👨‍💻 Professional notebooks 👨‍🚀

Hunter’s haphazard hoard of hopefully helpful hacks

Finances

Salary information

Information and comparison (levels.fyi)

Equity compensation

Overview
Tax implications

AWS

General

AWS services explained in one line each
Overview of AWS services (10 hour video)
AWS regions and corresponding AZs
- aws ec2 describe-availability-zones --region us-west-1 check currently-available AZs for region
Tools
- cloud-nuke delete all resources in an AWS account, optionally by region

VPC

Overview

EC2

Instance type naming conventions
Instance types
Purchasing options

IAM

Identities (users, groups, roles)

Kubernetes

General

Reference
Tools
- kubectx + kubens easily switch contexts and namespaces
- Popeye quickly profile cluster
- Stern tail logs from multiple pods
- helm-docs auto-generated Helm chart readmes
Snippets
- kubectl get secret orc8r-certs -o go-template='{{range $k,$v := .data}}{{printf "%s:\n" $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}' print a map of opaque secrets
Miscellanea
- Controller vs. operator: controller is generic reconciliation loop (e.g. blue-green deployment), operator is a controller that manages a particular application (e.g. Postgres operator)

API server

Object identifiers
- Type: specific Go struct defining the schema of an object, e.g. Pod struct
- Version: point-in-time identifier for representation of a type, e.g. v1 or v1beta1
- Kind: unique name for the type within the version, e.g. Pod or Service
- Group: set of related versions, kinds, and types that evolve together over time, e.g. core or apps
- API Version: combination of group and version to specify the API to target, e.g. v1 (i.e. core/v1) or apps/v1beta1
Object versioning
- Overview: object updates always update resource version, and only update generation if the update changes the spec
- Resource Version: opaque versioning string for optimistic concurrency control (backed by etcd revisions)
- Generation: sequence number incremented when the object’s desired state is changed

Scheduling

Pod scheduling cycle
Resource requests and limits

Node runtime

Scale

Scale limits for single cluster
- 100 pods per node
- 5k nodes
- 150k pods
- 300k containers

DNS resolution

EKS

Architecture

Docker

Containers

About (wikipedia)
Overview (playlist)
What is a container: namespaces and cgroups
- And union mounts (intro) (history)
Container vs. VM
Container vs. unikernel (about)

Reference

Dockerfile reference
- Prefer COPY over ADD
- Build cache rules overview
  - ADD and COPY only commands to dynamically invalidate
  - ARG change invalidates line of first use
  - RUN only invalidates if command to run changes
Logging overview – output of container’s PID 1 is forwarded to host as the application’s logs
Docker-compose networking
Tools
- lazydocker: TUI for Docker
- dive: explore Docker image and layer contents
- gosu: Docker-targeted su/sudo implementation for dropping a process’s privilege level

Snippets

docker network inspect -f '{{json .Containers}}' orc8r_default | jq '.[] | .Name + ":" + .IPv4Address' print all IP addresses in a Docker network
docker cp ~/Downloads/out.sql orc8r_postgres_1:/ copy file to Docker container
docker exec -it orc8r_maria_1 mysql -u magma_dev -p exec command with args in container (e.g. bash)
docker ps list containers
docker system df what’s available to prune
docker system prune docker doesn’t perform garbage collection
docker-compose logs -f controller* follow logs for controller container
docker-compose ps list containers (in context of docker-compose.yml)
docker-compose ps --services list services

Runtime privilege and Linux capabilities

Default granted
- AUDIT_WRITE write records to kernel auditing log
- CHOWN make arbitrary changes to file UIDs and GIDs (see chown(2))
- DAC_OVERRIDE bypass file read, write, and execute permission checks
- FOWNER bypass permission checks on operations that normally require the file system UID of the process to match the UID of the file
- FSETID don’t clear set-user-ID and set-group-ID permission bits when a file is modified
- KILL bypass permission checks for sending signals
- MKNOD create special files using mknod(2)
- NET_BIND_SERVICE bind a socket to internet domain privileged ports (port numbers less than 1024)
- NET_RAW use RAW and PACKET sockets
- SETFCAP set file capabilities
- SETGID make arbitrary manipulations of process GIDs and supplementary GID list
- SETPCAP modify process capabilities
- SETUID make arbitrary manipulations of process UIDs
- SYS_CHROOT use chroot(2), change root directory
Default not granted
- AUDIT_CONTROL enable and disable kernel auditing; change auditing filter rules; retrieve auditing status and filtering rules
- AUDIT_READ allow reading the audit log via multicast netlink socket
- BLOCK_SUSPEND allow preventing system suspends
- BPF allow creating BPF maps, loading BPF Type Format (BTF) data, retrieve JITed code of BPF programs, and more
- CHECKPOINT_RESTORE allow checkpoint/restore related operations. Introduced in kernel 5.9
- DAC_READ_SEARCH bypass file read permission checks and directory read and execute permission checks
- IPC_LOCK lock memory (mlock(2), mlockall(2), mmap(2), shmctl(2))
- IPC_OWNER bypass permission checks for operations on System V IPC objects
- LEASE establish leases on arbitrary files (see fcntl(2))
- LINUX_IMMUTABLE set the FS_APPEND_FL and FS_IMMUTABLE_FL i-node flags
- MAC_ADMIN allow MAC configuration or state changes. Implemented for the Smack LSM
- MAC_OVERRIDE override Mandatory Access Control (MAC). Implemented for the Smack Linux Security Module (LSM)
- NET_ADMIN perform various network-related operations
- NET_BROADCAST make socket broadcasts, and listen to multicasts
- PERFMON allow system performance and observability privileged operations using perf_events, i915_perf and other kernel subsystems
- SYS_ADMIN perform a range of system administration operations
- SYS_BOOT use reboot(2) and kexec_load(2), reboot and load a new kernel for later execution
- SYS_MODULE load and unload kernel modules
- SYS_NICE raise process nice value (nice(2), setpriority(2)) and change the nice value for arbitrary processes
- SYS_PACCT use acct(2), switch process accounting on or off
- SYS_PTRACE trace arbitrary processes using ptrace(2)
- SYS_RAWIO perform I/O port operations (iopl(2) and ioperm(2))
- SYS_RESOURCE override resource Limits
- SYS_TIME set system clock (settimeofday(2), stime(2), adjtimex(2)); set real-time (hardware) clock
- SYS_TTY_CONFIG use vhangup(2); employ various privileged ioctl(2) operations on virtual terminals
- SYSLOG perform privileged syslog(2) operations
- WAKE_ALARM trigger something that will wake up the system

Tcpdump

Overview
docker exec orc8r_controller_1 tcpdump -U -w - 2>/dev/null -- > controller.pcap tcpdump to file on host machine
Capture container network traffic, from host, with no extra per-container installs
- Create container to capture traffic sh docker build -t tcpdump - <<EOF FROM ubuntu RUN apt-get update && apt-get install -y tcpdump CMD tcpdump -i eth0 EOF
- docker run -i --net=container:orc8r_controller_1 tcpdump tcpdump -U -w - 2>/dev/null -- > controller_and_proxy.pcap capture directly onto host

Cloud

Prometheus

Snippets
- {__name__=~".+"} query all non-stale metrics
Architecture
Alternatives
- OpenTSDB: push-based, sharded TSDBs, HBase as the storage layer
  - TCollector: runs local collector plugins, then pushes metrics to the TSD; default collectors are analogous to Prometheus’s node exporter

Terraform

Reference
- Intro
- Official tutorials
- Debugging: set TF_LOG environment variable to one of TRACE, DEBUG, INFO, WARN, ERROR
Tools
- tfenv Terraform version manager
- terraform-docs generate module documentation
Snippets
- chomp(file("~/.magma/github_access_token")) remove newline from file input

Elasticsearch

Architecture
Scale limits for single domain (AWS)
- 3 PB storage (default limit is 600 TB)
- 200 instances (default limit is 40 instances)

Swagger

Tools
- Swag generate swagger.yml from godoc comments
- go-swagger better generated Go code
  - Model generation rules
  - swagger mixin merge Swagger specs using go-openapi/analysis

Interviews

Overview

Interview types

Interviews generally fall into 1 of 3 categories

Coding: design and code up a solution to a single problem (coding pad)
- DS&A: most coding interviews will require data structures and algorithms (DS&A) insights to complete
- Practical: a minority of coding interviews will just focus on practical, day-to-day coding skills
- Frontend: if you’re going for a frontend role, you will likely be challenged to build a UI component of some sort
Design: explore and design a high-level architecture for an under-specified problem (virtual whiteboard)
- System: most design interviews will require designing a scalable, efficient, robust distributed system
- Object-oriented: some design interviews will focus on the design of a single, in-process component
- Product: for more frontend or product management type roles, the focus will be on designing an API, and what UX to provide
Behavioral: describe approaches and past experiences (verbal discussion)

Additional notes

Timing: most companies will do 1 hour per interview, where 50-55 minutes are devoted to the actual interview; some however will only allot 45 minutes per interview, where 35-40 minutes are devoted to the interview (e.g. Meta)
Number of questions/parts: most companies only expect 1 question/part per interview; some however expect multiple (potentially completely separate) questions within a single interview (e.g. Meta expects 2 separate questions during coding interviews)
Split: behavioral questions will mostly be standalone in their own interview; increasingly I’ve started seeing the behavioral component split out into the first 10-15 minutes of each of the other interviews
Idiosyncratic interview types
- Bug squash: set up an IDE in a repo and resolve as many bugs as you can (e.g. Stripe)
- Technical deep dive: this can either be a discussion-only system design interview (e.g. LinkedIn), or a project retrospective where you present the technical details of a past project (e.g. OpenAI)
- Project design: read a project’s technical requirements, then write up a design doc-style one pager on your design recommendations (e.g. Waymo)
- Technical trivia: answer trivia questions on a technical topic (e.g. Jump Trading)

Tools

Use levels.fyi to set level and compensation expectations per company
Design interview tools (to move faster / save time)
- I recommend defaulting to Excalidraw and using the largest iPad Pro you can buy, with an Apple Pencil, as an additional display (it can be a little buggy, but I personally still like it)
- At minimum, use a mouse, rather than a trackpad

Remote-friendly

Remote-friendly big-tech companies (as of January 2024)

Meta (IC6 and above)
Microsoft
Netflix
Pinterest
Stripe

Prep

Rubrics

Coding
- Communication
- Problem solving
- Coding
- Verification
Design
- Exploration
- Handling data
- Component responsibilities
- Completeness
- Tradeoffs
- Quantitative
- Deep dive
Behavioral
- Leadership
- Problem solving
- Teamwork
- Communication
- Impact
- Values

Coding

Resources
- LeetCode top 150

Design

See the system design section below for more info/examples
Resources
Response template
- Explore (5 min)
- Estimate (5 min)
- Think (2 min)
- Design goals (2 min)
- High-level design (5 min)
- Deep dive (20 min)
- Wrap-up (walkthrough, and if time: further considerations, additional trade-offs (5 min)

Behavioral

STAR method: Situation, Task, Action, Results
- Organize responses using this paradigm
Ask ChatGPT to be a behavioral interviewer
- For the target company and level
- Tell it to give feedback on each response: score out of 100 and how could improve
- Use speech-to-text mode – you talk, it transcribes then responds with text

System design

Specific problems

Rate limiting (intro) (distributed)
Load balancing (intro)
Full text search (SSTable + LSM tree)

Partitioning and replication

Partitioning techniques
- Consistent hashing: place bins and balls onto a circle (with virtual bins)
- Rendezvous hashing: compute w(i, j) = \text{hash}(\text{ball}(i), \text{bin}(j)), assign ball to the k bins with highest weights
- Optimal placement (e.g. Meta’s Shard Manager): place shards according to application-reported metrics and goals, look up shard location in consistent datastore

Latency

Software engineering

Overview

Career framework (and comparable framework from Dropbox)

Decision making

Steps toward making a decision
1. Assess the situation (understand initial question + facts)
2. Get input (clarify key question + assemble factbase)
3. Develop alternatives
4. Recommend an alternative
5. Decide
6. Communicate decision
7. Execute
RAPID framework: each letter is a specific role, with the Decider as the DRI

Development

High quality software is worth the cost (Martin Fowler)
How to write a design doc
Productive remote development
- iTerm2 rather than Terminal.app
- Eternal Terminal rather than SSH
- tmux in native mode, e.g. alias e0="et -c 'tmux -CC new-session -AD -s main' -x devvm7602.ftw0.facebook.com:8080"
- Remote pbcopy to copy from remote to local
- iTerm2 triggers to send local desktop notifications on remote command completions, e.g. with this noti command

User experience

SLOs, SLAs, SLIs
Errors and user control
- Mistake (error in planning or rule application), lapse (failure of memory), slip (failure of execution)
- Capture slip: started the same way, veered off into wrong action set
- Description slip: chose wrong action set, because it’s similar to the desired action set

Collaboration

Wandbox: online IDE for many languages and versions, with shareable permalinks
CodeWithMe: IntelliJ IDE sharing
tmate: terminal session sharing

Patterns

General
- Modern software development (12-factor app)
- Level-triggered vs. edge-triggered (state vs. event)
- Broker vs. brokerless architectures
- Dependency injection (DI) allows a caller to supply the dependencies of a callee, supporting e.g. easier testing
- Data access object (DAO) abstract, often restricted interface around a database or other persistence mechanism
Domain-driven design
- Domain primitives
  - Set domain primitives, e.g. Quantity or DeliveryDays as an object with validation
  - Perform validation either on creation or as early as possible, e.g. at API boundaries
  - Result in “if an object exists, it contains a valid value”
Security
- Payload validation order: origin, size, lexical content, syntax, then lastly semantics
API design
- List pagination: page size, page token, and next page token
Testing
- Nil package-level function run during a method, to be set by testing functions, allowing testing functions to run custom hooks during the real method
Code quality
- 80-character line lengths aren’t optimal anymore

Culture

Concurrency

Typing

Static vs. dynamic typing: static checks types before run-time, dynamic check types on-the-fly during run-time
What’s wrong with null: non-composable, requires dynamic vs. static type checking

Tools

Code quality
- LGTM static analysis on PRs
- Codacy configurable linting on PRs
- Codecov test coverage
- golangci-lint Go linters aggregator
Code navigation
- Sourcegraph universal code search, with code coverage plugins
- Sourcetrail source code visualization
Security
- SOPS store encrypted secrets in version control
Packaging
- goreleaser package and deliver Go binaries
Utilities
- csvtomd convert CSV file to Markdown table
- Regex101 full-featured regex tester

Libraries

Go
- Trending Go repositories
- List of Go projects (Awesome Go)
- Afero file system abstraction
- Cobra CLI generation
- Ent ORM for graph-based entities
- Errors extends standard errors library with support for context wrapping
- Gorm default ORM
- Sqlx extensions to database/sql
- Viper configuration
Python
- List of Python projects (Awesome Python)
- CFFI interface with C/C++ code
- Click CLI generation
- Fabric execute shell commands remotely over SSH (intro)
- MonkeyType generate static type annotations by collecting runtime types
- Nubia interactive shell generation
- NumPy storage and operation on large, multi-dimensional arrays
  - About (home) (reference)
  - Visual guide
  - Tutorial
  - Cheat sheets (pdf) (html)
  - Operators
    - A @ B matrix multiplication of A by B
- Swig generate bindings for C/C++ code
- XAR package many files into a single self-contained executable file

SSH

Intro (wikipedia) (overview) (examples)
Resources
ssh command
- -vvv max verbosity
- ssh vagrant@localhost -p 2222 specify non-standard port
- Vagrant-specific
  - vagrant ssh-config ssh configuration for each VM, e.g. which private key to use
  - vagrant ssh cwag_test -- -vvv send SSH options through vagrant command
Configuration
- System-wide config files
  - ~/.ssh/known_hosts public key info for hosts known to the client (file format) (key format)
    - Fields: markers (optional), hostnames, keytype, base64-encoded key, comment, space-delimited
    - Keytype may be further decomposed, e.g. into bits, exponent, modulus
- Connection config sources, by order of precedence (common options)
  - Command line-specified
  - ~/.ssh/config
  - /etc/ssh/ssh_config
- Config for transient servers, e.g. local VMs
  - UserKnownHostsFile=/dev/null don’t pollute user’s known_hosts file
  - StrictHostKeyChecking=no don’t enforce known_hosts entries
  - IdentityFile indicate which private key to use for host
Public key access (howto)
- Create public-private keypair
- Install non-secret public key in ~/.ssh/authorize_keys on target server
- Indicate location of private key on local host, if it’s in a non-standard location
Message ID values

IntelliJ

IntelliJ IDEA
IdeaVim, Vim emulation for IntelliJ IDEs
Set up automatic code formatting (Go)
Fixes
- Prevent auto-closing tabs: Settings > Editor > General > Editor Tabs > Tab limit
- Remote interpreter will choose cached version of files, workaround is to add local version of the files
Easy actions
- Contextual menu > implement interface
Keymaps
- Miscellanea
  - shift shift omni-action
  - <alt-enter> contextual menu
  - <ctrl-tab> switcher
  - ctrl ctrl run anything
  - <cmd-up> navigation bar
- Remote targets
  - <cmd-alt-y> sync file
  - Invalidate Caches / Restart nuke caches
- Go to
  - <cmd-b>, <ctrl-j> declaration
  - <cmd-shift-b> type declaration
  - <cmd-k> implementations
  - <cmd-shift-k> implements
  - <cmd-j> show usages
  - <cmd-shift-j> find usages
  - f2, <ctrl-;> next error
  - <shift-f2>, <ctrl-'> previous error
  - <ctrl-shift-;> next change
  - <ctrl-shift-'> previous change
  - <ctrl-.> next method
  - <ctrl-,> previous method
- View
  - <shift-esc> hide active tool window
  - <cmd-shift-'> maximize tool window
  - <cmd-shift-e> recent locations window
  - <cmd-3> find window
  - <cmd-e> open recent files
  - <cmd-t> terminal window
  - <cmd-o> reveal in file tree
  - <cmd-+> expand all in project menu
  - <cmd-shift--> close all folds
  - <cmd-shift-=> expand all folds
  - <cmd-y> quick definition
  - <cmd-shift-a> version control annotations
  - <cmd-shift-d> file/package structure
- Search
  - !*.go exclude-based file mask
  - right click > mark directory as > excluded exclude specific directories
- Edit
  - <shift-f6> refactor
  - <shift-f6> <shift-f6> refactor dialogue (e.g. disable string and comment refactor)
  - <shift-n> create new file
  - <c-n> next autocomplete option
  - <c-p> previous autocomplete option
  - <cmd-l> roll back changed hunk
- Run
  - <cmd-r> run
  - <cmd-shift-r> rerun
- Move
  - <cmd-shift-t> open tests
  - <alt-tab> cycle split
  - <alt-shift-tab> cycle previous split
  - <ctrl-cmd-n> split vertically
  - <ctrl-cmd-m> split horizontally
  - <ctrl-cmd-z> unsplit
  - <ctrl-cmd-y> unsplit all
  - <ctrl-cmd-o> move tab to opposite split

Serialization

Intro

Some formats

Protocol buffers (protobufs) generically usable
FlatBuffers (flatbufs) zero-parse
JSON (JavaScript Object Notation) untyped, non-binary
ASN.1 (Abstract Syntax Notation One) multiple encodings, canonical encoding available

Protocol buffers

Message definition forward compatibility
Encoding is ID-tag-value, requiring protobuf descriptor to correctly deserialize
- ID unambiguously defines the type of the field by referencing the associated protobuf definition
- Tag values (wire types) determine length of the field
  - 0 varint – int32, int64, uint32, uint64, sint32, sint64, bool, enum
  - 1 64-bit – fixed64, sfixed64, double
  - 2 length-delimited – string, bytes, embedded messages, packed repeated fields
  - 5 32-bit – fixed32, sfixed32, float
- Value is encoded depending on the type
  - Varint fields are encoded least-significant bits first, with the MSB of each non-terminal byte as 1, else 0
  - Length-delimited fields are encoded as length-value
  - 64-bit and 32-bit fields are fixed-width
Style guides (Google) (Uber)
- Common issues
  - Consistent service namings
- Example file
- File structure
  - License
  - File overview
  - Syntax
  - Package
  - Imports (sorted)
  - File options
  - Everything else
Go
- protobuf/ptypes/wrappers provides wrapper structs for Go built-in types, e.g. StringValue, BytesValue, etc.
- proto.Equal(protoA, protoB) compare equality of protobufs
- proto.Clone(data).(*protos.SubscriberData) clone a protobuf
- Sending any.Any
- Alternative gRPC encodings and compressors

gRPC

Awesome gRPC gRPC and protobuf resources
Streaming RPCs strongly discouraged (about) (40m video)
protoc --go_out=plugins=grpc:. *.proto generate gRPC-supported protobuf code
Go
- Hello world
  - go mod init github.com/hcgatewood/any-json init Go modules to import third-party and cross-package code in toy project
- Returning nil, errors.New("zebra") for a response results in the specified error returned to client
- Returning nil, nil for a response results in an error (error while marshaling) returned to client, even if the return message is defined as empty

Canonical forms

Overview
- Seems like the best solution to deriving a usably-canonical form (aside from defaulting to ASN.1) is picking a particular JSON canonicalizing format and accordingly converting relevant objects to JSON
- Protobufs support a canonical encoding into JSON, so this seems like the best way to convert protobufs into a canonical representation
Protobufs
- Protobufs provide no cross-language canonical serialization form
- Some languages (e.g. Go, C++) provide deterministic, but not canonical, serialization
- One of a set of issues blocking a canonical format is map key-value pair ordering is undefined
  - In Go, map ordering seems randomized across serialization invocations
JSON
- No single, RFC-like standard describing a canonical JSON form
  - matrix.org (Python)
  - gibson042 (Go)
- Seems like Go’s Protobuf JSON encoder jsonpb produces more-or-less canonical JSON
- Seems like Python’s general JSON encoder json can be coerced into producing more-or-less canonical JSON

Literature

Great books
- Meta list of great books lists
- Mortimer Adler’s recommended reading list, from How to Read a Book
- Harold Bloom’s western canon, splits into four ages: theocratic, aristocratic, democratic, and chaotic
Other reading lists
Miscellanea
- NASA’s science fiction space technology terms

Catholicism

Games

Traditional sports

Football overview

Factorio

Factorio cheat sheet

Scrabble

Official dictionary
Two-letter words (full list) (with definitions)
- AA, AB, AD, AE, AG, AH, AI, AL, AM, AN, AR, AS, AT, AW, AX, AY
- BA, BE, BI, BO, BY
- DA, DE, DO
- ED, EF, EH, EL, EM, EN, ER, ES, ET, EW, EX
- FA, FE
- GI, GO
- HA, HE, HI, HM, HO
- ID, IF, IN, IS, IT
- JO
- KA, KI
- LA, LI, LO
- MA, ME, MI, MM, MO, MU, MY
- NA, NE, NO, NU
- OD, OE, OF, OH, OI, OK, OM, ON, OP, OR, OS, OW, OX, OY
- PA, PE, PI, PO
- QI
- RE
- SH, SI, SO
- TA, TE, TI, TO
- UH, UM, UN, UP, US, UT
- WE, WO
- XI, XU
- YA, YE, YO
- ZA

Housing

General

Useful
- List of places to live (Ask HN)
- City guides (Wikivoyage)
- Rent calculator
- Square feet visualizer
- When moving to a new location, look into getting a third-party company to provide rental tours
Apps to make life easier
- Instacart grocery delivery
- Getaround: car share
- Transit: live public transportation trip planning, including bikes and electric scooters
- Lemonade: renter’s insurance

San Francisco

Recreation
- Where to play basketball
Relevant maps

Linear algebra

Resources

Introduction to Applied Linear Algebra by Boyd, Vandenberghe (“VMLS”)
- Corresponding Julia Language Companion
Introduction to Linear Algebra, fifth edition by Strang (“Strang”)
- Corresponding lecture series
Mathematics for Computer Science by Lehman, Leighton, Meyer (“6.042”)
- Corresponding lecture series
Matrices cookbook
Applications of linear algebra
- Original Google Page Rank algorithm
Further study
- Linear algebra
  - Further linear algebra study
- Abstract algebra
  - Lecture series (Harvard Mathematics 122)

Topics

Associativity of matrix multiplication
Dot product intuition
Orthogonality implies linear independence, but not the other way around
Disambiguation of “affine” vs. “linear”
Projections (intro) (as linear transformation)
Eigenvalue visualization
Volume of a generalized cone
Common sets and number types
- \emptyset = \{\} the empty set
- \mathbb{N} = \{0, 1, 2, \ldots\} nonnegative integers aka natural numbers (sometimes not including zero)
- \mathbb{Z} = \{\ldots, -2, -1, 0, 1, 2, \ldots\} integers
- \mathbb{Q} \supset \{\frac{1}{2}, -\frac{5}{3}, 16\} rational numbers
- \mathbb{R} \supset \{\pi, e, -9, \sqrt{2}\} real numbers
- \mathbb{C} \supset \{i, \frac{19}{2}, \sqrt{2}-2i\} complex numbers
- Modifiers
  - The + and - superscripts restrict a set to its positive and negative elements, respectively
  - \mathbb{Z^*} = \{0, 1, 2, \ldots\} nonnegative integers
Inverse of a 2x2 matrix
Eigenvalue equalities \begin{aligned} \prod \lambda_i &= \det A \\ \sum \lambda_i &= \sum a_{ii} \end{aligned}
Projections \begin{aligned} A\hat{x} &= b \\ \hat{x} &= (A^TA)^{-1}A^{T}b \\ p = A\hat{x} &= A(A^TA)^{-1}A^{T}b \\ P &= A(A^TA)^{-1}A^{T} \\ P^2 &= P \\ P^T &= P \\ A \text{ invertible} &\implies P = I \\ |P| &\in \{0, 1\} \end{aligned}
Inverse of matrix product \begin{aligned} (B^{-1}A^{-1})(AB) &= B^{-1}(A^{-1}A)B \\ &= B^{-1}B \\ &= I \\ \tilde{\implies} (AB)^{-1} &= B^{-1}A^{-1} \end{aligned}
Cumulative moving average \begin{aligned} a_{n+1} &= \frac{x_{n+1} + nc_n}{n+1} \\ \iff a_{n+1} &= a_n + \frac{x_{n+1} - a_n}{n+1} \end{aligned}
The four fundamental subspaces
Set relations and binary relations
- Function: right-unique and left-total
- Injective aka one-to-one: left-unique, may be a function
- Surjective aka onto: right-total, may be a function
- Bijective: injective and surjective, may be a function
Convex vs. affine combinations
- Affine combination: sum of coefficients = 1; coefficients may be negative
- Convex combination (“weighted average”): same, but coefficients >= 0

Julia

Reference

Snippets

using LinearAlgebra; Diagonal(v) diagonal matrix from vector v

Jupyter notebooks

“Jupyter” is an amalgam of “Julia”, “Python”, and “R”
About (home) (overview) (docs) (wikipedia)
Math equation rendering uses MathJax (usage)
Getting started
- Local
  - Install Python: brew install python3
  - Install Jupyter: pip3 install jupyter
  - Install Julia: brew cask install julia
  - Install IJulia, the Julia kernel for Jupyter notebooks (other kernels)
```
$ julia
julia> using Pkg
julia> Pkg.add("IJulia")
```
  - Start Jupyter server: jupyter notebook (from relevant directory)
  - Print as PDF: jupyter nbconvert --to pdf in.ipynb
- Hosted
  - Use JuliaBox (docs)

Strength training

General

Form

All
- Appropriately brace for each set/rep
Deadlift
- Intro (Jeff Nippard)
- Avoid starting with hips too low
- Avoid rounding back during lift (push floor away)
- Avoid swaying the bar during lift (engage lats)
- Avoid jerking the bar (lift upper back)
- Avoid overextending during lockout
Hip thrust (barbell)
Overhead press
- Flex glutes
Squat
- Allow knees to go past toes
- Explode out of the hole

Fixes

Posture support
- Rounded shoulders
  - Stretch minor pec
  - Wall shoulder girdle stretch
  - Strengthen pull motion (rowing exercises)
- Lordosis (anterior pelvic tilt)
  - Strengthen core, glutes, hamstrings
  - Stretch hip flexors (e.g. using bench)
- Forward head posture
  - Isometric lying neck retraction: lie on back with feet flat on ground, then hold a push of the head into mat; relax and repeat

Macro tracking

Lifesum

Kinesiology

Kinesiology reference and glossary
Anatomical planes
- Sagittal: left and right
- Coronal: front and back
- Transverse: top and bottom
Anatomical terms of location
- Bipeds vs. quadrupeds
- Dorsal toward belly, ventral toward back
- Anterior toward front, posterior toward back
- Proximal toward main mass of the body, distal away
- Medial toward center, lateral toward sides
Anatomical terms of motion
- List of joints and their movements
- Muscular contractions
  - Concentric resulting in its shortening, eccentric during its lengthening
  - Dynamic resulting in movement, isometric without significant movement
- Joint motions
  - Flexion decreases joint angle, extension increases
  - Abduction moves away from center of body, adduction toward
  - Rotation: external rotates away from center of body, internal toward
  - Rotation: pronation results in upward facing, supination results in downward facing
Anatomical structure
- Spinal column

Shell

Intro

General

Startup scripts (reference)
- ~/.profile for any login shell, e.g. setting environment variables
- ~/.bashrc for interactive Bash usage, e.g. aliases
- ~/.bash_profile for Bash-specific login shells
- Exception: for Mac’s Terminal.app, each new terminal window runs a login shell
Path
- hash: common commands can be added to hash lookup, superseding normal path lookups
  - Indication: type rm outputs e.g. rm is hashed (/bin/rm)
  - hash -d vim reload hash table entries for vim
  - hash -r reload all hash table entries
Readline: line-editing and history for Bash and other command-line interfaces (wikipedia)
- Commands (assuming vi mode)
  - Cheat sheet (vi mode) (emacs mode)
  - ctrl-r, ctrl-s reverse/forward substring search
  - v edit current command in editor, then execute (:cq from Vim to abort)
- Options
  - set editing-mode vi set vi mode for all Readline-dependent tools
- Expansions
  - !* all previous aruments
  - !^ first of previous aruments
  - !$ last of previous aruments
  - !2 second of previous aruments
  - !2-3 second and third of previous aruments
History (intro)
- history -r read history file and append contents to history list
- history -a append new history lines to history file
- history -s add args to end of history list as single entry
- History expansion (stackoverflow)
  - !:1 first argument of previous command
  - !:1-2 first and second arguments of previous command
  - !$ last argument of previous command
  - ^cat^bat run previous command with cat replaced to bat
Redirection (cheat sheet)
- openssl x509 -help 2>&1 | less redirect stderr to stdout
- cat <(echo 1) redirect stdout to anonymous fifo, then pass fifo to cat as an argument
Completion
- Bash doesn’t automcatically handle completion for aliases (Zsh does, however)
- complete -p git get complete’s completion handling for git
Exit status
- Exit code values
  - 0 indicates successful command
  - 1-255 indicates failed command, generally acts as an error code
- echo $? print exit status of most recent command
Kill a running process via increasingly drastic measures
1. ctrl-c send SIGINT
2. ctrl-\ send SIGQUIT
3. ctrl-z send SIGSTP to suspend program
4. kill kill the process ID from another terminal
5. kill -9 force-kill the process ID from another terminal, without a clean abort
Miscellanea
- Debug scripts by toggling debug mode via set -x and set +x
- Programmable completion (tab expansion)
- Functions can be exported via export -f
- pushd ., popd, dirs to cd via a stack of directories

Scripting

set -euf -o pipefail scripting settings for safer script exits
Heredoc syntax supports defining a separate file inline sh cat <<EOF foo bar EOF
Variable assignment
- VARIABLE=${1:-DEFAULTVALUE} provide default variable value
Control flow: loops
- for i in {0..3} ; do echo $i ; done loops 0 1 2 3
- for i in $(seq 3) ; do echo $i ; done loops 1 2 3
- for (( i = 0 ; i < 3 ; i++ )) ; do echo $i ; done loops 0 1 2
- Reading lines from file ```sh while read p; do echo “$p” done <peptides.txt
  
  # or
  
  cat peptides.txt | while read line; do echo “$line” done ```
Control flow: conditionals
- if [[ -d tmp ]] && [[ 1 == 1 ]] ; then echo 42 ; else echo 43; fi
Conditional expressions
- Comparing integers via arithmetic expression or conditional expression: both are fine
- Single vs. double square brackets: prefer double unless POSIX compatibility is strongly desired
- (($foo < $bar)) arithmetic
- [[ $foo -lt $bar ]] conditional
- [[ ! 42 == 42 ]] negate conditional
- Common expressions
  - [[ $foo == "foo" ]] string equality
  - [[ -d $foo ]] exists and is a directory
  - [[ -f $foo ]] exists and is a regular file
  - [[ -r $foo ]] exists and is readable

Snippets

Timed spinner

# spin for the passed number of seconds, defaulting to 10 seconds.
function spin() {
  nsec=$(( ${1-10}*10 ))
  spin='-\|/'
  for (( i = 0 ; i < nsec ; i++ )) ; do
    j=$(( i % 4 ))
    printf "\r${spin:$j:1}"
    sleep .1
  done
  printf "\r \n"
}

Options

set -o vi set vi mode just for shell
shopt -s histappend allow parallel shells to correctly append to the history file
stty -ixon fix forward search in some terminals

Tools

General
- htop interactive process viewer
Search
- fzf fuzzy finder (examples)
- rg line-oriented search
- rga seach PDFs, Office documents, and more
- scc count lines of code (similar to cloc, loc) (also: tokei, e.g. tokei -o json | tokei-pie to generate a recurisve pie chart)
Data wrangling
- fx JSON explorer (also: jless)
- visidata tabular data explorer (also: datasette)
Files
- ranger Vim-like file manager (also: broot)
  - ~/.config/rc.conf startup commands and key bindings
  - cw change current filename
  - S open shell in current directory
  - :bulkrename bulk-rename selection (select via shift-v)
  - :delete delete selection
  - :touch create file (:shell touch create multiple files at once)
  - :flat -1 flatten all directories under current (e.g. for easier bulk rename)
- rclone sync files and directories to and from local or cloud storage solutions
  - Supports Dropbox, Google Drive, FTP, HTTPS (and client-side certificate validation), and more
  - rclone serve http . serve local directory and files over HTTP
- watchman execute commands in response to file modifications (also: entr)
Network
- bandwhich network utilization by various categories
- iftop top for network usage
  - iftop -i en0 view overview for interface en0
Media
- easytag view and edit audio file tags
- ffmpeg audio/video processing (common commands)
  - Intro to video technology: image, video, codec, and more
  - ffmpeg -i input.wav output.mp3 convert WAV file to MP3
- imagemagick convert, edit, compose images
  - magick test.svg test.png convet SVG image to PNG
  - magick IMG_4281.PNG IMG_4282.PNG -quality 100 golds.pdf magick multiple images into a single PDF
  - magick drivers-license-2023.pdf -quality 100 drivers-license-2023.png magick a multi-page PDF into multiple images
  - magick mogrify -monitor -format png *.HEIC convert all HEIC images to png format
- youtube-dl download from YouTube, Soundcloud, Funimation, and others
Miscellanea
- export-chrome-bookmarks export Google Chrome bookmarks and history
- fortune random messages from a database (about)
  - $(brew --prefix)/Cellar/fortune/9708/share/games/fortunes location of fortune files (for Homebrew fortune installations)
  - % meta character separating each fortune
  - <tab><tab>-- Thérèse de Lisieux attribution format (and no " for quotes)
  - strfile catholic generate catholic.dat for consumption by fortune
- redshift adjust display color temperature according to surroundings

Miscellanea

Technical topics

Self-learning
Compilations
- Awesome lists
- Free tools for devs
- Software engineering blogs
- The best software tech talks ever, ranked
- Best computer science papers (best paper awards)
- Programming aphorisms (Hacker Laws)
- Plain text tools
Articles
Classes and textbooks
- Underactuated Robotics: algorithms for walking, running, swimming, flying, and manipulation
- Introduction to algorithmic trading
Organizational tools
- Notion
- Airtable
Other resources
- Personal automation
  - Self-hosted solutions (deploy-your-own-sass) (awesome-selfhosted)
  - Home Assistant: open-source home automation
  - Tailscale, ngrok: expose services on local networks (firewall/NAT busting)
  - ntfy.sh: send notifications to your phone
- yEd Graph Editor: diagrams generation
- List of startups hiring for remote work
- List of European tech visas and work permits
- Awesome design tools
- Some standard document templates, e.g. pitch, vision, strategy, product requirements, etc.

Non-technical lists

Question lists
- List of controversial topics (ProCon.org)
- List of questions to get to know someone
Busy person patterns
- Just start
- Contiguous time blocks
- Single-task important items
- Put it off
- Seek clarification
- Batch the simple stuff
- Task jar
- Strike when you are hot
- Feedback loop
- Prioritize
- Delegate
- Drop unimportant tasks
- Good enough
How to be successful
- Compound yourself
- Have almost too much self-belief
- Learn to think independently
- Get good at “sales”
- Make it easy to take risks
- Focus
- Work hard
- Be bold
- Be willful
- Be hard to compete with
- Build a network
- You get rich by owning things
- Be internally driven

Life hacks

Append a plus sign to Gmail address: john+spam@gmail.com delivered to john@gmail.com, but can be filtered e.g. as if it was spam
Force a public Wi-Fi connection page to appear
- The “connection page” is a web page called a captive portal
- Often making an HTTP request will allow the network to insert the captive portal
- Debugging steps
  1. Direct a browser to http://www.example.com (or e.g. http://captive.apple.com for Macs), in Incognito mode if necessary
  2. Explore further debugging steps
Encode Wi-Fi access point password into a QR code

Quick links

Personal data access (Facebook) (Google)
Update third-party permissions (Facebook) (Google) (Dropbox) (GitHub) (Spotify) (Apple)
Manage recurring payments (PayPal)
Tax forms (USAA) (Coinbase) (Nelnet)
Free access to research papers via Sci-Hub (about) (where is Sci-Hub now)
Southwest (travel funds) (low fare calendar)
USPS change of address form
Web calendars
- Sports games (ESPN) (Google)
Public torrent sites
- List of torrent sites (list 1) (list 2)
- Usually just hosting magnet links with useful associated metadata, rather than full torrent files
- How magnet links work

Sell vehicle privately

Howto in TX
Listing
- Nice pictures of the washed car
- Extremely detailed – VIN number, trim, features, etc
- Forms of payment accepted
- Screencap of CARFAX and CARFAX (or Edmunds/KBB) valuation
Desired documents
- Service history
- CARFAX report
Required documents (take pictures of all items once completed)
- Title
- Title transfer from new state (in TX, and many other states, also functions as bill of sale)
- Bill of sale (if the title transfer isn’t also a bill of sale)
Point of sale
- Ensure all personal items removed from vehicle
- Take picture of buyer’s license, then do test drive (preferably where buyer just rides along, but permit test drive if desired)
- (Optional) accompany buyer for an independent mechanic’s inspection (which buyer pay for)
- Together, withdraw a certified check from the buyer’s bank
- Perform the switchover in the following order
  - Sign documents
  - Deposit certified check (ideally using mobile app)
  - Hand over documents
  - Remove identifying/personal info from vehicle
    - Plates
    - Registration sticker
    - Toll tags
    - Final check for personal items
  - Hand over keys
- (Optional) accompany buyer to county tax office to support title application submission
Followups
- Notify state of the transfer
- Cancel/transfer auto insurance

Food

Apple rankings
- Honeycrisp apples (91)
- Pink lady apples (80)
Beef grades

Home

Houseplant care overview
- Care sheets for specific houseplants (reddit) (bloomscape)

Tech

Power cables (NEMA and IEC)
USB connectors

Other

Useful apps and services
- Hipcamp: book camping experiences (Airbnb for campgrounds)
Clothing style
Leather grades (reddit) (summary) (wikipedia)
- Highest to lowest quality
  - Full-grain
  - Top-grain
  - Split-grain (e.g. suede)
  - Genuine (just a marketing term–not a real grade)
  - Bonded
- Can determine whether substance is real leather by running a lighter across its surface (leather is mildly resistant to burning/melting)
How to hang curtains
TV dimension and resolution calculator
Exploratory, descriptive, and explanatory research
1. Exploratory initiates look into a space, sans prior background research
2. Descriptive attempts to gain unbiased, generally numerical data on topics in the research area
3. Explanatory attempts to explain previous descriptive research, toward the goal of a more complete understanding of the research area
Flight dynamics
SI base unit
- meter: length
- kilogram: mass
- second: time
- ampere: electric current
- kelvin: temperature
- candela: luminous intensity
- mole: amount of substance
SI derived unit
Better emailing patterns

Grammar and style

Style

How to write in plain English
- Short sentences
- Active verbs
- Use “you” and “we”
- Avoid nominalisations
- Use lists
Use adverbs sparingly

Punctuation

Hyphen, en dash, em dash (Chicago Manual of Style)
- Hyphen (-) connects things that are intimately related, like “toll-free” or “two-thirds” (- in LaTeX)
- En dash (–) connects things over a range, like “May–September” or “chapters 3–5” (-- in LaTeX)
- Em dash (—) functions more like parenthesis, separating two mildly related parts of a sentence (--- in LaTeX)

Disambiguations

Method vs. methodology: a methodology does not define specific methods, but rather surveys and/or compares existing methods

Abbreviations

Latin abbreviations
Always use a comma after “i.e.” and “e.g.”
Don’t italicize “i.e.”, “e.g.”, or “etc.”
- Foreign words commonly accepted in English are not italicized (Quora)
- Common Latin abbreviations are set in roman (Chicago Manual of Style)

LaTeX

Overview

About
Overview of common tasks
Common errors
Full intro
Documentation
Package documentation
Full documentations listing
Finding symbol names (Detexify)
Word count via TeXcount: texcount -merge main.text

Intro

Installation
- Via MacTex: download and install
Document generation
- latexmk -pvc in.tex make file and open in PDF viewer (intro) (docs)
  - latexmk -c clean up temporary tex-related files
- Alternatively, just use Overleaf

Minimal .tex file

\documentclass[12pt]{article}
\usepackage[a4paper, total={6in, 8in}]{geometry}
\begin{document}
Hello world.
\end{document}

Miscellanea

LaTeX-style math rendering in a browser
- KaTeX: fast server-side or client-side rendering 🌟
- MathJax: full-featured client-side rendering
Style and quirks
- e.g.\ normal spacing don’t put wider space after a particular period
Math mode
- $x \text{ is equal to } 42$ normal text in math mode using the amsmath package
\textcolor{gray}{\xmark} change text color with the xcolor package

Citations

Intro
- Bibliography management with BibTeX and biblatex
Landscape of options
- File format
  - BibTeX (.bib)
- Interface between .bib file and LaTeX
  - BibTeX
  - Biber 🌟
- Package for citation formatting
  - natbib
  - biblatex (getting started) (cheat sheet) 🌟

Process

Reference management software like Mendeley creates a BibTeX file (.bib)
Reference the relevant BibTeX file; e.g. by creating a symlink in the local directory

Cite using the biblatex package

% Minimal example using biblatex
\usepackage[sorting=none]{biblatex}
\addbibresource{bibtex-file.bib}
% ...
\cite{AuthorYear}
\cite[pre-text][post-text]{Author1Year1,Author2Year2}
% ...
\printbibliography{}
% ...

Choosing a reference manager
- Favorite options
  - Mendeley 🌟
  - Zotero
  - Papers
- Comparison
  - Mendeley vs. Zotero (Quora) (Hacker News) (York University)
  - All options

General

Tikz (intro) (docs)

Examples

Tree

\begin{figure}[ht]
\centering
\begin{tikzpicture}[
    grow = left,
    sibling distance = 2em,
    level distance = 8em,
]
\node {payload}
    child {node {TLS}
        child {node {HTTP} edge from parent[<-,black]}
        child {node {IMAP/POP3} edge from parent[<-,black]}
        child {node {arbitrary application} edge from parent[<-,black]}
        edge from parent [<-,red]
    }
;
\end{tikzpicture}
\caption{Reuse of TLS as the mechanism for secure communication.}
\label{fig:tls-reuse}
\end{figure}

Including a personal macros file

Include in the preamble with \usepackage{hcg-personal}

% hcg-personal.sty
\NeedsTeXFormat{LaTeX2e}
\ProvidesPackage{hcg-personal}[2019/04/16 hcgatewood's personal style file]

% Be sure to use \RequirePackage instead of \usepackage
\RequirePackage{amssymb}

% ...

Hyperlinks
- Use the hyperref package, and always load it last (docs)
- \href{url}{text} link to url
- \url{url} link to url, without the representative link text
- \hyperref[label_name]{''link text''} link to label_name

Figures

% Include a centered picture of a gull, with caption
\begin{figure}[ht]
\centering
\includegraphics[width=0.5\textwidth]{gull}
\caption{A picture of a gull.}
\label{fig:gull}
\end{figure}

Tables

\begin{table}[ht]
\centering
\begin{tabular}{c|c|c|c}
\multicolumn{1}{c}{} & \multicolumn{1}{c}{\underline{Operator co-located}} & \\
& Agent & Controller & Witness \\
\hline \hline
Manned & \cmark & \cmark & \cmark \\
\hline
Unmanned & & \cmark / \xmark & \cmark \\
\hline
Logical swarm & & \cmark / \xmark & \cmark \\
\hline
Swarm constituents & & & \\
\end{tabular}
\caption{Comparing manned, unmanned, and swarm systems.}
\label{tbl:manned-unmanned-comparison}
\end{table}

Gantt charts

With pgfgantt

% Example calendar-based Gantt chart
\ganttset{calendar week text={\currentweek}}
\begin{ganttchart}[
    inline,
    x unit=0.7mm,
    time slot format=isodate,
]{2018-12-03}{2019-05-24}
  \gantttitlecalendar{year, month, week} \\
  \ganttbar{Synthesize}{2018-12-15}{2019-02-01}
\end{ganttchart}

Forest, for better trees

Example

\begin{figure}[ht]
\centering
\begin{forest}
    for tree = {minimum size=3em},
    internal/.style = {draw,circle},
    affected/.style = {internal,fill=lightgray},
    changed/.style = {internal,dashed,edge={dashed},for children={edge={dashed}}},
    leaf/.style = {draw,square,fill=black,text=white,font=\sffamily\bfseries},
    [k,affected
        [k14,affected
            [k12,internal
                [k1,internal [u1,leaf]]
                [k2,internal [u2,leaf]]
            ]
            [k34,affected
                [k3,changed [u3,leaf]]
                [k4,internal [u4,leaf]]
            ]
        ]
        [k58,internal
            [k56,internal
                [k5,internal [u5,leaf]]
                [k6,internal [u6,leaf]]
            ]
            [k78,internal
                [k7,internal [u7,leaf]]
                [k8,internal [u8,leaf]]
            ]
        ]
    ]
\end{forest}

Machine learning

Tools
- fastText efficient text classification and representation learning (tutorial)
Resources
- PracticalAI practical approach to machine learning
- Neural networks in JavaScript
Articles
- Lessons learned building an ML trading system that turned $5k into $200k

Distributed systems

Consistency (wikipedia)
- Simplified overview of consistency
- Sequential consistency vs. serializability: sequential consistency is unconcerned with interleavings or the possibility of interleavings, as long as each client’s operations are enacted in the order specified by the client
- Sequential consistency vs. linearizability: many possible/legal interleavings under sequential consistency, while linearizability forces a particular interleaving based on global timestamps
- Consistency models
Papers
- MapReduce paper
- Raft paper
- Raft state machine

Algorithms

Resources

Combinatorial optimization suite (Google OR-Tools)

Algorithms

Linear-time median finding
Reservoir sampling: choosing k items uniformly and efficiently from a list of n items, where n very large
Dynamic programming
- Seam carving: content-aware image resizing (intro) (improved)

Data structures

Visualizations
Bloom filter space-efficient probabilistic data structure reporting set membership (“no” or “probably”) (visualization)
- Cuckoo filter same function, often better in practice (visualization)
How to implement strings
Hexagonal grids
Rope: store and manipulate a very long string, e.g. by a text editor
Wave function collapse: take in an “archetypal” input, then use that implicit set of rules to produce procedurally-generated output; generall used for images or 3D designs

Probability

Prior probability vs. posterior probability
- Prior is without additional evidence
- Posterior is including evidence
Moments of a random variable:
- 1st moment: mean
- 2nd central moment: variance
- 3rd central moment: skewness
- 4th central moment: kurtosis
Making sense of standard deviation
Intuitive understanding of covariance
Poisson distribution
Exponential distribution (the only memoryless distribution)
Beta distribution: represents prior belief about a particular probability; allows recursive updating
Convolutions
Logarithm rules

Wolfram Alpha

P(n, k) k-permutations from n-sized set
C(n, k) k-combinations from n-sized set
k+1 where k=2 evaluate for some assigned variable

Pandoc

User guide
Markdown
- GitHub markdown emoji (full list) (rendered) (sorted subset)
Markdown to HTML
- Download a CSS style as pandoc.css
- pandoc --standalone --self-contained --css=pandoc.css --katex=katex_0.10.0/ --from markdown+smart+emoji in.md -o out.html convert in.md to standalone out.html
- Can upload directly to GitHub Gists and view the HTML via GitHub HTML Preview
Markdown to PDF
- pandoc --from markdown+smart --pdf-engine=xelatex -V urlcolor=blue in.md -o out.pdf
- pandoc --from markdown+smart --pdf-engine=xelatex --template eisvogel --listings in.md -o out.pdf using a template

Future topics

3D printing
- Onshape: 3D printing design tool
Software-defined radio (SDR)
- Intro

Raspberry Pi

Intro

Documentation: setup, installation, usage, etc.
Command cheat sheet
Awesome Raspberry Pi
Serial howto

Boards

Raspberry Pi
- Raspberry Pi 3 Model B+ $35 board
- Raspberry Pi Zero W $10 board with built-in Wi-Fi and Bluetooth
NanoPi
- NanoPi NEO Plus2 $35 board, similar to Raspberry Pi 3 Model B+
- NanoPi Duo $15 board with Wi-Fi

Physical components

Pinout reference (and for other related boards)
Power from GPIO pins: 5V to 5V, GND to GND, but no regulation/fuse protection against over-voltage and current spikes

Install Raspbian

Download Raspbian
Unzip and copy image to SD card
- Copy image to SD: dd if=2018-04-18-raspbian-stretch.img of=/dev/sdX bs=4M conv=fsync status=progress
- Single step: unzip -p 2018-04-18-raspbian-stretch.zip | dd of=/dev/sdX bs=4M conv=fsync status=progress
- Can also use cat instead of dd
- On some machines, had an issue with accessing the unzipped file–so just use the single step command
Safely remove SD card: sudo sync
Insert SD card into Pi, power on, plug in HDMI monitor and USB peripherals, boot
Edit desired config values in /boot/config.txt and reboot

Software

picamera: Python bindings for Raspberry Pi camera module
pigpio: circumvent requiring root for access to GPIO pins requires root
WiringPi: GPIO access, including hardware PWM
- For NanoPi boards, use WiringNP
- gpio command-line utility (requires root)
  - gpio readall view (somewhat idiosyncratic) pin number mappings
  - gpio mode 1 pwm set pin 1 to use PWM
  - gpio pwm 1 1023 set pin 1 to max PWM value
  - gpio mode 16 out set pin 16 to output mode
  - gpio write 16 1 set pin 16 to high
- Python bindings

Fun projects

General guides (intro) (learning) (all)
“Super computing” cluster

Peripherals

USB-to-UART adapter: Adafruit USB-to-TTL serial cable

Hardware

Raspberry Pi UARTs: disabled by default
RS-232 vs. TTL Serial (UART)
I2C: Inter-Integrated Circuit (intro) (in-depth)
Power supply: CV means “constant voltage”, CC means “constant current”
Level shifter vs. voltage regulator: level shifter converts one digital signal to another, voltage regulator produces constant output voltage from higher, mildly variable input voltage
How DC motors work, part of the intro to motors series

Miscellanea

Micro Center store in Boston
Assign static IP address: edit /etc/dhcpcd.conf then reboot
Armbian is a Debian-based Linux distribution (documentation)
Open window options, e.g. to resize: <alt><space>
Raspberry Pi-specific system information with vcgencmd (video core general commands): vcgencmd commands
Playing video requires hardware acceleration, provided e.g. by OMXPlayer (part of default Raspbian installation)
Yellow lightning bolt icon means under-voltage warning: try different USB power cable

Linux

General

Debian

General information, including network services
Official documentation
Systemd: a system and service manager, among other features

Ubuntu

Official documentation
All man pages, across versions
Get Ubuntu version: lsb_release -a
Change hostname persistently: edit /etc/hostname
Package management
- apt list --installed list installed packages
- Installing a newer version of Python
- Source packaging is required for Debian packages
- Supported hardware: i386, amd64, armel, and more
  - i386 vs. amd64: 32-bit vs. 64-bit of x86 architecture
Snaps: self-contained, transactionally-updated method for software distribution targeting IoT devices

Performance

Networking

Routing and traffic control howto
Configuring network interfaces
Interface configuration for ifup and ifdown
Parse host’s local IP address
- hostname -I list “all” resolvable addresses for the host
- ip -f inet addr show eth0 | grep -oP 'inet \K[\d.]+' get the IPv4 address on interface eth0 (\K means “output starting here”)
- Other options (SO 1) (SO 2)

Proxies

Set proxy location: something like the following in /etc/environment (username:password only as-necessary):

export http_proxy=http://username:password@proxyhost:port/
export https_proxy=https://username:password@proxyhost:port/
export ftp_proxy=http://username:password@proxyhost:port/

Run sudo commands through a proxy: add line Defaults env_keep = "http_proxy ftp_proxy" using visudo command

File system

Partition: logical section of a physical disk acting as a unique disk
Block devices and block sizes
- Block size used by file system
- Sector size used by disk/storage
- Block size may be a multiple of sector size
- Block device: hardware component storing blocks of bytes (e.g. hard disk rather than keyboard)
blockdev
- blockdev --getbsz /dev/sda get block size of sda
- blockdev --getsize64 /dev/sda get total size of sda
dd
- Can be used to create/restore/clone/transfer, rescue file with bad blocks, benchmark device throughput
fdisk
- fdisk -l | grep "Sector size" disk sector size
stat
- stat file information about file
- stat -f file information about filesystem on which file resides
- Three types of timestamps
  - atime: access, or last time file was read
  - mtime: modify, or last time file content was modified
  - ctime: change, or last time file metadata was modified
- Understanding timestamps
  - Use touch to update timestamps
  - atime is generally not straightforward, often replaced with relatime
Permissions

systemd

Intro
Snippets
- sudo journalctl print all logs
- sudo journalctl -f follow logs tail
- sudo journalctl -u jcaus print all logs for jcaus unit
- sudo systemctl enable ~/some.service enable from unit file at arbitrary location
Unit file
- Assign unit an alias: alias=jcaus.service in the [Install] section
- Minimal example, starts at boot time
  1. Create unit file
```
# /etc/systemd/system/big-old-test.service
[Unit]
Description=a big old test

[Service]
ExecStart=/bin/bash -c "while true; do date; sleep 1; done"

[Install]
WantedBy=multi-user.target
```
  2. Reload systemd: sudo systemctl daemon-reload
  3. Enable service: sudo systemctl enable big-old-test
  4. Optionally, start right away: sudo systemctl start big-old-test

Commands

Overview and list
dmesg read/write kernel messages, e.g. to check Ethernet cable connection and status
lsblk list block devices (type of device file), where e.g. mmc refers to a MultiMediaCard
lscpu display information about the CPU architecture
lsusb list USB devices
mmv move and rename files in bulk
passwd change password
ping send ICMP ECHO_REQUEST to network hosts (ping6) (ping vs. ping6)
pwgen generate passwords
ssh secure login and shell
strace trace system calls and signals (cheat sheet)
sysctl view and configure kernel parameters at runtime
tar save many files as a single file
time time a simple command
uname print system information like OS, kernel name, machine hardware name
vimdiff view useful diffs between two files (cheat sheet)

Snippets

#!/usr/bin/env bash most portable hashbang
command gist twilio.js call non-aliased version of gist
cp foo{,bar} expands to cp foo foobar
date +%s number of seconds since the Epoch
date --set='Fri Nov 22 23:36:40 UTC 2019' set system time
date -u date and time in UTC
dhclient eth0 configure interface eth0 with dynamically-assigned IP
dmesg -w like tail -f for dmesg
du -s . | awk '{print $1}' get recursive size of current directory in bytes
echo -e 'hello\nworld' print literal newline (can also use printf)
echo foobar | gzip | zcat pipe directly to gzip
env print environment variables
find . -exec touch {} \; recursively change date on all files in directory
find . -type f -name '*.go' -print0 | xargs -0 sed -i '' 's/MapMany/Map/g' recursively replace in all files
grep -RI foobar . recursively search all text-like files for foobar
gsed 's/\x1b\[[0-9;]*m//g' < terraform_apply.txt > terraform_apply.log remove ANSI color codes from text
hg status | sed -n 2p | awk '{print $2}' get line 2, column 2 of hg status output
ip -f inet addr show eth0 | grep -oP 'inet \K[\d.]+'
man 2 unshare view the man page for unshare from section 2
mdfind -name .txt -onlyin ~/magma/ macOS version of find using Spotlight index
netstat -tulnp list ports and determine what process is bound to a port
nmap -p - localhost localhost port scan
nohup livedown start notebooks_main.md --port 8080 & start background command to last past terminal session
openssl rand -hex 20 generate random password
pbpaste | jq pretty-print copied JSON (replace pbpaste with appropriate clipboard manager)
ps -o ppid= 50377 get parent PID of PID 50377
ps aux | grep -i net case-insensitive search for processes matching net
rename 's/foo/bar/' \* rename files by replacing all occurences of foo with bar
route -n show routing information like gateway and genmask
sleep 4 sleep for 4 seconds, or use m, h, d for minutes, hours, days
sort favorite_words.txt | uniq -d print non-unique lines
sudo -E python3 preserve environment variables when sudo-ing
sudo hwclock --set --date 'Fri Nov 22 23:36:40 UTC 2019' && sudo hwclock --hctosys update hardware clock
sudo ncdu -x -o out.ncdu / && ncdu -f out.ncdu device disk utilization TUI
tail -f out.log | ag 'foo|bar' --passthru highlight streamed log file for foo and bar (case-insensitive)
tail -f out.log | egrep 'foo|$' -i --color highlight streamed log file for foo with grep (case-insensitive)
tar czf target.tar.gz file1 file2 file3 create gzipped tar file
tar xzf source.tar.gz extract gzipped tar file
textutil -convert txt main_tf_08_06_2020.rtf convert to/from plaintext
unset COMPOSE_FILE unset environment variables
watch ip addr watch and wait for IP changes
xxd -p < /dev/input/event1 convert binary data from event1 to printable format

Miscellanea

Namespaces isolate access to global system resources (overview) (man)
Network interfaces and network namespaces
I/O buffering
- Buffering rules for stdout
  - stdin: line-buffered
  - stdout (TTY): line-buffered
  - stdout (not a TTY): fully-buffered
  - stderr: unbuffered
- Change a stream’s default buffering mode programmatically using setvbuf
- Force stdout to be line-buffered:
  - Programmatically: setlinebuf(stdout);
  - From shell using stdbuf: stdbuf -o L grep jcaus /var/log/syslog
Set tab completion for shell alias:
1. complete | grep systemctl
2. complete -F _systemctl sc
Point-to-point connection over Ethernet (on Ubuntu): assign static, optionally private IP addresses to each machine
Sysbench: benchmark OS parameters
Pseudo file system: on Linux /proc holds process-specific information, /sys exports kernel objects
Install VMWare Fusion with Ubuntu on a Mac
Read device events (from e.g. USB devices) with evdev using libevdev (Python bindings with the evdev module)
gzip vs. zip: tar is archiver, gzip is compressor; zip does both in one step
Tarball: collection of files archived as a single file (from Tape ARchive)
Locks
- Semaphore vs. lock/mutex: only the thread that locked a mutex is supposed to unlock it
- Mutex vs. spinlock: mutex puts thread to sleep when not available, while spinlocks have thread keep retrying

Version control

General

Copybara transforming and moving code between repositories

Git

General
Tools
- diff-so-fancy cleaner diffs
  - ^(Date|added|deleted|modified|renamed): search term to jump between changed files
- gh official GitHub CLI
- git-quick-stats generate statistics, e.g. _GIT_BRANCH=master git-quick-stats --git-stats-by-branch
- hub GitHub wrapper
- scm_breeze easier Git file references
- tig TUI
Commands
- Reset
  1. Move the branch HEAD points to (--soft)
  2. Make the index look like HEAD (default)
  3. Make the working directory look like the index (--hard)
- Diffs
  - git diff remotes/origin/master --name-only only names of changed files
- Rewriting history
  - git commit --amend rebase/replace most recent commit (and message)
  - git rebase -i HEAD~3 interactive rebase of most recent 3 commits (HEAD~3 actually points to the parent of the last commit to update)
  - git push origin my-branch -f force push to my-branch
- Branches
  - git branch -a list all branches (local and remote)
  - git branch -d local-branch delete local-branch
  - git checkout --track remotes/origin/remote-branch checkout and track remote-branch
  - git merge other-branch merge other-branch into current branch
  - git checkout -b new-branch a9c146a create new-branch from commit
- Searching
  - git log --oneline -S hosting-helm-private commits which changed the occurrences of the string
GitHub
- GitHub Actions workflow events, e.g. pull_request, pull_request_target, etc.
- git-history view file changes across commits
- Chrome extensions
  - refined-github overall better GitHub experience
  - notifier-for-github desktop notifications

Web programming

Chrome DevTools

Console utilities
- $('#header') return first element with ID header
- $$('img') return all img elements
Net internals
- chrome://net-internals/?#dns query, flush Chrome’s DNS resolution
Snippets
- $$('.check').forEach((e) => e.click()) click all elements with class check

NPM

Docs
Change NPM global install location
Snippets
- npm install -g git+https://github.com/hcgatewood/livedown install (global) package directly from GitHub

DOM

Edit

JavaScript

Chart.js: HTML5 charts

REST

Microsoft REST API guidelines

HTTP

HTTP/2 uses a binary protocol to multiplex multiple HTTP requests over the same TCP connection (intro) (spec)
URLs
- Specification (RFC 3986)
- URI := scheme:[//authority]path[?query][#fragment]
- authority := [userinfo@]host[:port]
- URI, URL, URN: URL identifies a resource by its (current) location, URN uniquely identifies a resource by its name, and URI is the union of the URL and URN types
How tracking pixels work
HTTP caching
- Overview
- Disable browser caching (Chrome) (Safari)
- Shared vs. private caching
- Cache-Control header

Page caching (bfcache)

Different form of caching than HTTP caching
Detect via pageshow and pagehide window events
Disabling the page cache

// Reload the window when page loaded from page cache
pageshow = function (event) {
  socket.emit('ready')
  if (event.persisted)
    window.location.reload()
}

// Hide the body when leaving page to prevent
// "blinking" on navigation back/forward to page
pagehide = function (event) {
  // The '$' operator can be from e.g. zepto.js
  $('body').css('opacity', '0')
}

// Install listeners
// - pageshow: any time the page is loaded, whether from
//      page cache or otherwise
// - beforeunload: any time the page is destructed, whether
//      from forward/back or closing
window.addEventListener('pageshow', pageshow)
// window.addEventListener('pagehide', pagehide)
window.addEventListener('beforeunload', pagehide)

Operating systems

Overview

Interactive map of Linux kernel

Reference

MIT 6.828: Operating System Engineering
OSDev: operating systems wiki
Intel 80386 Reference Programmer’s Manual: manual for the original i386 microprocessor, the earliest in the x86 family
Writing an OS in Rust (1st edition) (2nd edition)
Linux kernel in-depth (A Heavily Commented Linux Kernel Source Code)
What every programmer should know about memory
Example OS’s
- xv6: didactic, written in C for x86
- Redox: written in Rust for x86
- TNeo: RTOS, written in C for ARM microcontrollers (overview)
- Linux: written in C, cross-platform (topics)
Assembly
Libraries
- lwIP: lightweight TCP/IP stack
Virtualization
- Overview (from CMU) (from Harvard)
- QEMU docs
- QEMU networking
Hardware
- Overview (Modern microprocessors - a 90-minute guide)
- Intel’s E1000 data sheet
Collection of mostly low-level topics
Patterns
- Read-copy-update (RCU): use atomic pointer reads/writes to enforce coherent data structure views, and many-reader/single-writer synchronization to determine when an old object can be freed

General

Memory addressing

mov (%ebx), %eax         /* load 4 bytes from memory address in ebx into eax */
mov %ebx, var(,1)        /* move contents of ebx into the 4 bytes at memory address var */
mov -4(%esi), %eax       /* move 4 bytes at memory address esi+(-4) into eax */
mov %cl, (%esi,%eax,1)   /* move contents of cl into the byte at address esi+eax */
mov (%esi,%ebx,4), %edx  /* move the 4 bytes of data at address esi+4*ebx into edx */

Trap vs. interrupt: trap is an interrupt generated in a user process (software-generated exception); an interrupt in general can be asynchronously generated by hardware
PHY: circuitry connecting the link layer device (e.g. a wider Ethernet NIC) to the physical medium
Memory-mapped vs. port-mapped I/O
CMOS, aka nonvolatile BIOS memory: stores e.g. system time, BIOS settings
Recursive page tables (in xv6) (general description)
Virtual to physical address translation:
Translation registers:
Interrupt description table:
Stack during subroutine call:
Inode (disk inode in xv6):
Network server environments in JOS (we implemented green boxes):
Processor vs. CPU:

Golang

Overview

Intro
- Tour of Go
- Writing, building, installing, and testing Go code
Reference
Project layout
- Standard project layout
Install
- Goenv, like pyenv but for Go, uses shell shims
Style
- Advice (Practical Go)
- Google style guide (Go code review comments)
  - Error strings lowercased
  - Initialisms consistently-cased, e.g. ID rather than Id
  - Interface generality: if a type exists only to implement an interface, the type should be private and a constructor function returns the interface
  - Delcaring empty slices
- Generated code
  - A Go file is assumed to be generated if any of the lines before the first non-comment, non-blank line match the regex ^// Code generated .* DO NOT EDIT\.$
- Miscellanea
  - Prefer &Obj{} over new(Obj) where possible, as the struct literal is newer and idiomatic for applicable use-cases

Language

Performance
- Debugging performance issues in Go programs
Concurrency
- Patterns
- Data race detector
- Channel implementation
- How goroutines work
- How to combine goroutines with sync.WaitGroup
- Communicating sequential processes (Bell Labs and CSP Threads): inspiration for Go’s concurrency model
Memory
- Pointer arithmetic not allowed
- Memory allocation
- Nil
Strings
- Strings, bytes, runes, and characters
- Back quotes (backticks: `) encase non-escaped string literals

Standard libraries

context cross-boundary context cancelation (intro)
encoding/json
- Overview
- Streaming encoders/decoders
  - Wrapper around io.Reader, io.Writer
  - enc := json.NewEncoder(<file>) create encoder
  - err := enc.Encode(&v) encode v to file
fmt formatted I/O, including print
- %v default format, e.g. for structs
- %+v add field names for structs
- %#v print in full Go syntax
- %T type of value
net/rpc
strconv string conversions
strings

Testing

Overview
- Testing patterns
Notes
- Inter-package tests are run concurrently by go test, by default

Tools

panicparse parse panic stacks into human-readable text
stress runs a test until it fails (install with go install golang.org/x/tools/cmd/stress@latest)

Miscellanea

Go is object-oriented: interface implementation and embedding provide subtyping (is-a) and object composition (has-a) relationships between structs and interfaces
Nil-safe getter for nested structs, as in generated protobuf code, checks if the receiving struct pointer is nil
Different types of nil (Understanding Nil)
// Deprecated: use alternative... mark object as deprecated

Snippets

go test -count=1 ./services/state/... force no-cache test run
import assert "github.com/stretchr/testify/require" have failed assertions end the test immediately
go test -c ./services/lte/obsidian/handlers/ && while ./handlers.test -test.run=TestAPNResource -test.count=100 -test.failfast ; do date ; done run flaky test till failure

Mac

Specific apps

Terminal
- Install Solarized theme by downloading Solarized and installing new profile in Terminal options
Apple Calendar
- View all events in specific calendar
  1. Search .
  2. Deselect all other calendars
Spotify
- Fix ineffective “forget device” (reset Spotify Connect device list)
  1. Log in to Spotify
  2. Go to “apps” page, remove access for Alexa
  3. Go to “account overview” page, click “sign out everywhere”
  4. On mobile, go to Alexa app > Settings > Music & Podcasts > Spotify, disable and re-enable Spotify skill
- Fix cached local files issue
  1. Close Spotify
  2. Remove ~/Library/Application Support/Spotify/Users/hcgatewood-user/local-files.bnk
  3. Reinstall Spotify

General

Peripheral ports
Options
- Set the primary display on multi-display setup by moving the white bar in system preferences > displays > arrangement
- sudo mdutil -a -i off disable Spotlight indexing
Understanding common processes found in the Activity Monitor
File system
- ~/Library/Application Support user-specific application storage
Troubleshoot issues with AirPods or other Bluetooth devices
1. Forget and re-pair the device
2. Update computer’s operating system 🌟
3. Reset Bluetooth module then restart computer: shift-alt-click Bluetooth icon > Debug > Reset Bluetooth module
4. Factory reset AirPods: hold down AirPods case button for 15 seconds
5. Try other potential solutions
Markdown
- markdown-toc --maxdepth 2 notebooks_main.md | pbcopy generate the table of contents for this file using markdown-toc
Convert text to speech stored as an audio file
- Copy the desired text (or save to text file)
- pbpaste | say -o out.aiff convert to speech stored as AIFF format
  - pbpaste pastes from the system clipboard
  - -r 250 set output to speak 250 words per minute
  - -f in.txt rather than taking from the clipboard, pull from in.txt
- Open out.aiff using e.g. VLC
  - VLC allows strong control over playback speed

Run a service at login

Using launchctl (overview) (docs)

Example launchctl plist file

<!-- ~/Library/LaunchAgents/com.user.login.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.user.login</string>
    <key>ProgramArguments</key>
    <array><string>/Users/Hunter/.login.sh</string></array>
    <!-- <key>StandardOutPath</key> -->
    <!-- <string>/Users/Hunter/.login.out</string> -->
    <!-- <key>StandardErrorPath</key> -->
    <!-- <string>/Users/Hunter/.login.out</string> -->
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>

Example login script

# ~/.login.sh
PATH=/usr/local/bin:/usr/bin:/bin
livedown start '/Users/Hunter/Dropbox (Personal)/notes/notebooks_main.md' --port 8081

Keyboard shortcuts

Intro
- Remap any keys, e.g. with Karabiner-Elements
- Remap switch control and caps lock keys
General
- Multi-touch gestures
- cmd-ctrl-q lock mac
- ctrl-down app expose
- ctrl-up mission control
- fn-up page up
- fn-down page down
- cmd-b open Spotlight contents as a search engine query
- cmd-ctrl-space emoji search
Finder
- cmd-shift-. show hidden files in Finder
- cmd-shift-g go to folder, with tab complete
Terminal
- alt-click move cursor location to mouse pointer
- cmd-2click open URL
- cmd-shift-2click select URL or file path
Screenshots
- cmd-shift-3 screenshot whole desktop
- cmd-shift-4 screenshot specified region
  - Before selection
    - space specific window
  - During selection
    - space hold to move chosen area
- cmd-shift-5 screenshot options and recordings
Chrome
- cmd-shift-t undo closing tab
YouTube
- , back one frame
- . forward one frame
Toggle connection to Bluetooth device (also consider ToothFairy)
- BluetoothConnector to wrap Bluetooth device control
- BetterTouchTool to run BluetoothConnector as a shell script
- /usr/local/bin/BluetoothConnector 00-00-00-00-00-00 --notify, substituting desired device’s MAC address
Vimium
- Commands
  - ? help, with option to show all/advanced commands
  - yy yank current URL
  - P paste clipboard in omnibox of new tab
  - gu go up URL hierarchy
  - gU go to root of URL hierarchy
  - gi focus first text input on the page
  - yf copy link URL
  - yt duplicate current tab
  - W move current tab to new window
  - >>, << move tab to the right/left
  - ]], [[ go to next/previous page (e.g. for Google search results)
- Visual mode
  1. Use find mode to select starting point
  2. c to enter caret mode (optional)
  3. v to enter visual mode
  4. y, Y to yank selected text
- Preferred mappings map H previousTab map L nextTab map J goForward map K goBack map ' Marks.activateGotoMode map \ passNextKey normal

Homebrew

Documentation
Related: mas, a CLI for the Mac App Store
Snippets
- brew update-reset hard update to fix any potential merge issues
- brew cleanup remove all old formulas
- brew update && brew upgrade update Homebrew, then upgrade all outdated packages
- brew deps --installed list top-level formulas (seems to not be working…)
Debugging
- z will sometimes stop working and cause abort trap: 6, fix with brew reinstall gawk

Essential apps

Other lists (Nikita Voloboev) (Awesome Mac)
Built-in
- Digital Color Meter
Amphetamine: prevent entering sleep mode
BetterTouchTool: keyboard shortcuts to move and resize windows, run shell scripts, etc.
Cleanup.pictures: fantastic tool to quickly clean up pictures
Codeshare: free, shared online codepad (alternative: Collabedit)
🌟 Excalidraw: free online sketching and diagramming, with collaboration
Flow: Pomodoro timer to automate work/break cycles (keyboard shortcuts)
F.lux: adjust display color temperature depending on location and time of day
Flycut: clipboard manager to store and paste up to 99 stored snippets at a time
HackMD collaborative Markdown, and can use a GitHub repo as backing store
Howler Pro: timer with notifications to set and record sprints through the day
NoPaste: online serverless pastebin
🌟 Obsidian: Markdown-based note taking app with Vim-mode, sync, and publish features
Read Aloud: browser extension to convert web page text to spoken audio
SequenceDiagram.org easy, shareable sequence diagrams
Vimium: Vim-like shortcuts in Google Chrome

Python

Intro

Installs

pyenv: switch between versions of Python, locally or globally, by injecting shims into the PATH (commands)
- pyenv version currently set Python version
- pyenv versions all installed, pyenv-selectable versions
- pyenv install --list list Python versions available for installation
- pyenv install 3.4.10 install version as a pyenv-selectable version
- pyenv local show version local to current working directory
- pyenv local 3.4.10 set local version for current working directory
- pyenv local --unset unset local version
- pyenv global 3.7.4 set global (per-user) version
Install pip
- Overview
- Pip on Ubuntu: sudo apt-get install python-pip python-dev build-essential
- Pip3 on Ubuntu: sudo apt-get install python3-pip
Install IPython: python3 -m pip install ipython
Install Python modules: python3 -m pip install some-package

Style

Avoid Hungarian notation, i.e. don’t include type information in an identifier’s name

General

Related tools
- MicroPython lean Python implementation targeting microcontrollers and constrained environments
Language miscellanea
- Docstring format python def some_function(): """A default one-line docstring.""" pass
- Unpacking (splat) operator
- Floats
  - m, e = math.frexp(4.2); m * 2**e == 4.2 exactly (extract float’s mantissa and exponent)
  - a, b = (4.2).as_integer_ratio; 4.2 == a / b exactly
- String formatting
  - '{:5.5}'.format('xylophone') truncates and pads to exactly xylop
- open modes
  - r read
  - w write (truncate first)
  - a append
  - r+ read/write
  - b binary (don’t decode file’s bytes contents to str)
- Async
  - await asyncio.gather(f(), g(), h()) run function calls concurrently
  - async for elt in awaitable_iterable iterate over async iterable
- Dotted notation for dictionaries
  - Via SimpleNamespace for the most simple needs
  - Via namedtuple under limited circumstances
- Type annotations (cheat sheet)
  - Prefer base classes where applicable
    - Iterable for generic iterables (anything usable in for)
    - Sequence where a sequence (supporting len and __getitem__) is required
  - Support for generics
  - Example ```python from typing import Callable, Union, Optional, List
    
    def f(num1: int, my_float: float = 3.5) -> float: return num1 + my_float
    
    x: Callable[[int, float], float] = f
    
    # Optional variables are allowed to be None def send_email( address: Union[str, List[str]], sender: str, cc: Optional[List[str]], bcc: Optional[List[str]], subject=’’, body: Optional[List[str]] = None ) -> bool: ```
Modules
- Multiple imports of the same module will only execute module-level code a single time
- The __all__ = [] construct determines what happens on import * from
Python bytecode: Python code compiled to bytecode for execution by the Python interpreter
- Cached as .pyc files (structure) (create)
- Interpreting the output of dis.dis(func)
- Patching bytecode (not production-recommended)
Socket programming howto
Strftime reference
- a abbreviated weekday
- W week number of the year
- -m month as decimal number
- -d day of the month as decimal number
- X time, with seconds
Logging cookbook
- Serializing log records from multiple processes
  - Recommended method is setting up a single process as a logging server
  - Clients make requests to the server, which then acts as the record serializer
  - Note that this socket-based approach is not thread-safe on the client’s side (and zerorpc doesn’t allow cross-thread sharing at all, so a different client-side logger would need to be used per-thread)
PDB, the Python debugger
- import pdb; pdb.set_trace() enter PDB programmatically
- pp obj.__dict__ pretty-print an object as a dict
Testing
- Overview (Hitchhiker’s Guide to Python)
- Testing tools taxonomy
- unittest, the standard library’s testing framework
  - Test classes subclass unittest.TestCase
  - Individual tests (methods on the subclass) begin test
  - setUp, tearDown, setUpClass, tearDownClass to handle before/after every and before/after all
  - debug instead of run to allow debugger usage and original exception propagation
  - Support for async tests (requires >= Python 3.8)
  - Automatic test discovery
  - Optional test skipping
- unittest.mock, the standard library’s mocking library
  - Mocks
    - Mock records calls and can be stubbed
    - MagicMock also autogenerates magic methods like __str__
    - AsyncMock returns awaitables
  - Patching
    - Monkey patch objects (functions, classes, contexts), e.g. to replace with mocks

Snippets

Point = collections.namedtuple('Point', ['x', 'y']); p = Point(11, y=22) create a named tuple
assert a == b, 'Something went wrong' prescribe an assertion-failure message
except (IndexError, ValueError) as err catch multiple possible errors
format(0xdeadbeef, '032b') format to binary number with leading zeros
from pprint import pprint import default pretty printer
print(sys.argv[1:]) command-line arguments
python3 -m http.server start simple HTTP server

Code

Run asynchronous main function (requires Python 3.7+)

import asyncio

async def main():
    """Asynchronous main function."""
    print('hello')
    await asyncio.sleep(1)
    print('world')

if __name__ == '__main__':
    asyncio.run(main())

Decorator to indicate and check parent method is being overridden

def overrides(interface_class):
    """
    Allow runtime checking of @override() decorator.

    Usage:
    > @overrides(logging.Handler)
    > def emit(self, record):
    >     pass

    See https://stackoverflow.com/questions/1167617/#answer-8313042.
    """

    override_err = 'Overriden method does not exist in parent.'

    def overrider(method):
        assert method.__name__ in dir(interface_class), override_err
        return method

    return overrider

Virtual machines

Intro

Vagrant

Use case: define and interact with development environments
Virtualization options include Docker, VirtualBox, VMware, and others
Plugins
- vagrant-scp SCP between host and VM
  - vagrant scp magma:/var/opt/magma/certs/gateway.key ~/Desktop/magma_protos/gateway.key ; vagrant scp magma:/var/opt/magma/certs/gateway.crt ~/Desktop/magma_protos/gateway.crt
Snippets
- vagrant ssh magma -c 'cd magma/lte/gateway && make test' run command over SSH
- vagrant box list list current images
- vagrant box add magma_dev_tmp magma_dev_virtualbox.box add image from Packer output
- vagrant cloud auth login login using info from $ATLAS_TOKEN
- vagrant cloud auth whoami who I’m logged in as
- vagrant cloud publish fbcmagma/magma_dev 1.0.1586316960 virtualbox magma_dev_virtualbox.box publish new version

Packer

Use case: define and build automated machine images
Notes
- Waiting for SSH to become available... can take 5-10 minutes
Snippets
- packer build magma-dev-virtualbox.json build image from config

Databases

General

Database engines ranking, with additional info
Interoperability and standardization: don’t assume full interoperability, though compatibility is possible for a subset of expressions across well-chosen SQL engines
Transaction isolation levels in common databases (Hermitage)
Best practices
Case sensitivity for identifiers
- Postgres converts unquoted to lowercase
- MySQL does not change unquoted
Tools
- gh-ost: online schema migrations for MySQL

Graph databases

Gremlin/TinkerPop tutorial

SQLite

Cheat sheet
How to corrupt an SQLite database file
Tools
- litecli: autocomplete and syntax highlighting for SQLite CLI
Snippets
- .help show help
- .tables list tables
- .schema ZRECIPE show a table’s schema
- .backup backup.sqlite create a consistent backup of the DB

Redis

Cheat sheet
Snippets
- redis-cli -p 6380 connect to Redis locally on port 6380
- keys * find all keys
- get mobilityd:systemd_status get value of key mobilityd:systemd_status

MySQL

Cheat sheet
MariaDB: drop-in-replacement fork of MySQL (comparison)
Command-line snippets
- mysql -u magma_dev -p access as user magma_dev, with password prompt
Snippets
- show databases;
- use magma_dev; use magma_dev database
SQL snippets
- show tables;
- describe some_table; show schema for table
- table some_table; print full table
- select * from some_table limit 10;
- select count(*) from some_table; number of rows in some_table
- select count(distinct key) from certificate_info_db; number of distict values in column key
- select id,convert(data using utf8) from reservations; convert a blob field data into readable UTF-8
- select id,convert(data using utf8) from reservations where data like '%testHardwareAssetIds%' limit 1; search e.g. a JSON field for particular key
- delete from certificate_info; delete all rows
- delete from states where type='directory_record' and key='id=foo'; delete based on column contents
- drop table some_table;
- begin transaction isolation level serializable; begin serializable transaction
- commit; commit transaction
- end; rollback transaction

PostgreSQL

Cheat sheet
PostgreSQL vs. MySQL
Job queue with SKIP LOCKED (more)
- Won’t work / doesn’t make sense with SERIALIZABLE isolation level
Dump and import
- Dump
  - Install postgresql-client version to match server
  - pg_dump -U magma -h orc8r.remote.host magma > /tmp/pgdump-prod-$(date +%s).sql dump remote magma database contents
- Import
  - createdb -U magma_dev -T template0 pgdump-prod-1598488376 create new virgin database
  - psql -U magma_dev pgdump-prod-1598488376 < /var/lib/postgresql/data/pgdump-prod-1598488376.sql import dumped database into created database
  - Note that may also have to create appropriate users/roles for imported database
- Drop
  - dropdb -U magma_dev pgdump-prod-1598488376 drop database
CLI snippets
- psql -U magma_dev access as user magma_dev, with password prompt
Snippets
- \l list all databases
- \c magma_dev use magma_dev database
- \dt list all tables in current database (\dt+ for more info)
- \d+ hwid_to_hostname detailed information on table hwid_to_hostname
- explain select task_id from test_task_bag limit 1 for update skip locked; explain query plan
- select key, encode(value, 'escape') as value from directoryd_blobstore where type='hwid_to_hostname'; decode bytes to string using encode string function
- select * from cfg_entities \g out.table output query to file
- \copy (select * from cfg_entities) to out.csv csv header output query to CSV file

C++

Google C++ style guide
constexpr indicate a variable or function value can be evaluated at compilation time

C

Intro

Resources
Style: (format) (portability) (header files)

Language

C operator precedence: highest precedence are ++ -- () [] . ->
static keyword inside a function: visible in function’s scope, but extends lifetime across all calls to function (and stored in the usual data or bss sections)
main function signature: int main (int argc, char *argv[], char *envp[])
- argc number of command-line args
- argv null-terminated array of C strings representing the passed command-line args
  - First arg: running program name
- envp null-terminated array of environment variable strings
  - Each take the format of "variable=value"
  - Less portable to include envp
  - Stale to the time of process fork
  - Not necessary, less usable than getenv
Error handling
- perror(char *msg): print msg and error message corresponding to errno
- exit(EXIT_FAILURE) portable exit with failure status
  - Call functions registered as on-exit, flush and close streams, remove tmp files
Print individual bits
Memory barrier, memory fence: enforce ordering constraint on memory operations (valuable for concurrent programs)
Labels are function-local by default
Define a small function in header file with static inline modifier
Declare a struct with typedef: typedef struct { int x, y; } Point;
Size of an array, also works for variable-sized arrays: int arr[arr_size]; sizeof(arr);
Zeroize a struct (initialization also works for static-sized arrays)
- On initialization: struct vertex_t vertex = {0};
- On-demand: memset(&vertex, 0, sizeof(vertex_t));
Type punning preferred with unions (or memcpy)
isatty: test whether file descriptor refers to a terminal
Avoid zombie processes: wait(), double fork(), or ignore in parent with signal(SIGCHLD, SIG_IGN)
How to write shared libraries
Libevent: event notification library for event-driven programming
Protothreads: lightweight, stackless “threads” for use especially in embedded systems

Standards

C standard
- Aka ANSI C, ISO C
- Standardized definition of the language, as jointly published by the ISO and IEC
- Recent versions include (C18) (C11) (C99)
- Where to find specific versions
  - Full text of C99 standard (html) (pdf)
Standard library
- C standard library is the standard library specified in ISO C (headers)
  - Aka ISO C library, libc
  - glibc is the prominent implementation of libc (source)
- POSIX: Portable Operating System Interface (headers) (wikipedia)
- POSIX is a superset of ISO C that also defines methods for interacting with the OS

Compilation

Compilation process: (source code) (assembly) (object code) (executable)
ELF: Executable and Linkable Format: standardized, cross-platform file format for executables, object code, shared libraries, and core dumps
GCC: GNU Compiler Collection: compiler system handling languages like C and C++
How the #include directive works: quotes first search current working directory, otherwise "" and <> are largely similar
echo | gcc -xc -E -v - print the include paths used by gcc/g++
gcc --print-file-name=libc.a look for a library, e.g. libc.so or libc.a
Cross compiling

Preprocessor

Reference (GCC’s C preprocessor)
Pragmas: provide additional information to the particular compiler, beyond language-standardized directives
- #pragma once modern alternative for providing #include guards (stackoverflow)
Current function name: __func__
Variadic macros: macros with a variable number of arguments
- Use a trick to send zero arguments to a variadic macro
Trigraphs: nine triplets, each beginning with ??, which can optionally be replaced by their alternative single-character representations; e.g. ??!??! represents ||, the logical or operator

Stringification with #

 // Will actually print EXP
 #define WARN_IF(EXP) \
 do { if (EXP) \
         fprintf (stderr, "Warning: " #EXP "\n"); } \
 while (0)

 WARN_IF(x == 0);  // prints "Warning: x == 0"

Token concatenation with ##

#define macro_start i ## n ## t m ## ain(void)
macro_start;  // -> int main(void);

Token expansion: in function-like macros, if either of the tokens next to a ## is a parameter name, it is replaced by its actual argument before ## executes

Debug print:

#ifdef DEBUG
#define DEBUG_PRINT(...) fprintf( stderr, __VA_ARGS__ )
#else
#define DEBUG_PRINT(...) do { } while (0)
#endif  // DEBUG

Make

Intro
Reference
Usage
- Automatic variables
  - $* the stem which an implicit rule matches
- Double-colon rules run a single matching rule instead of all matching rules for a target
- .PHONY targets if a file with the target’s name exists, prefer the target over the file
  - Can append via successive .PHONY designations
Snippets
- make -j make non-dependent executables/libraries in parallel
- make --debug=v print debug info
- $(PROTO_LIST): %_protos: build_list static pattern rule
- Self-documenting Makefile make help: ## Print help text @grep -Eh '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
- Print all variables make printvars: @$(foreach V,$(sort $(.VARIABLES)), \ $(if $(filter-out environment% default automatic, \ $(origin $V)),$(warning $V=$($V) ($(value $V)))))

CMake

Tutorial
Set up a debug build
- From CMakeLists.txt: set(CMAKE_BUILD_TYPE Debug)
- From command line: cmake -DCMAKE_BUILD_TYPE=Debug <...>
Hook up CMake with Vim’s YouCompleteMe via a compilation database
1. set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
2. Allow YCM to find the compile_commands.json file by either moving the file into some directory above the working directory (as in the link above), or utilizing a .ycm_extra_conf.py to point to the file
Remove unused header includes with include-what-you-use, which can be integrated with CMake

Sockets

Overview (Beej’s Guide to Network Programming)
glibc manual
Debugging
- If recv is mysteriously reading zero bytes, despite peer not closing connection, make sure the provided buffer length is non-zero
- Frozen program may be holding a socket
  - Check for whether local port is bound: nmap -p - localhost
  - Force-kill the parent process: kill -9 <pid>

GDB

Overview

The GNU Project debugger, allows start, stop, lookaround, and modification
Supports languages like C, C++, Go, Rust
rr: related debugger with deterministic replay and reverse execution

Intro

Cheat sheet
Intro (Beej’s Quick Guide to GDB)
Tutorial
Documentation
Intel’s x86 architecture manual
Better GUI options
- gdb-dashboard: in-place .gdbinit upgrade (doesn’t play well with gdb’s TUI)
- gdbgui: browser-based, full-featured

Commands

Startup

Pass command-line args
- From command line: gdb --args ./executable arg1 arg2 arg3
- From gdb: (gdb) r arg1 arg2 arg3
Automatically run executable: gdb -ex run --args ./executable arg1 arg2 arg3
Core dumps
- Message at program termination: Segmentation fault (core dumped)
- Load core dump file: gdb path/to/binary path/to/core
- Fix missing core dump, can be caused by issue on Ubuntu: ulimit -c unlimited
.gdbinit init script: run commands on gdb startup
- Can be placed in home directory or in current working directory
- Fix for occasional issue with loading .gdbinit from local dir
Interrupt execution programmatically
```
#include <signal.h>
raise(SIGINT);
```
Save command history across sessions: echo 'set history save on' >> ~/.gdbinit && chmod 600 ~/.gdbinit

Moving around

<C-c> interrupt execution
c continue execution
si step a machine instruction (s for source instruction)
ni same as si but step over function calls (n for source instructions)
finish run until current function finishes
return discard current stack frame and return immediately
break 120 if x == 3
- 120 can be <line>, <file>:<line>, <function>
- if x == 3 can be any conditional, or omitted
- info break info on all breakpoints
- disable 3 disable breakpoint 3
- enable 3 enable breakpoint 3
- del 3 delete breakpoint 3
advance 120 run until reach line 120 (or some location)

Looking around (tools)

General information
- wh open TUI
- bt stack backtrace
- list surrounding source lines
- frame current line, function, and args; change stack frame
- up/down move up/down stack frame
- info
  - Examining the symbol table
  - i r current registers
  - i address <symbol> address of <symbol>
  - i variables variables known in current scope
  - i locals local variables
  - i args args to current stack frame
  - i thread running threads
x examine memory
- x print raw contents of memory
  - x/nfu addr
    - n repeat count
    - f display format: hex, decimal, unsigned decimal, octal, two (binary), address, char, float, string, zero-padded hex, raw
    - u unit size: byte, halfword (2 bytes), word (4 bytes), giant word (8 bytes)
  - x/s <addr> print full null-terminated string starting at <addr>
  - x/5i $eip print next 5 instructions w/ next instruction at top
p(rint) print evaluated c expression
- p/x $cs print content of $cs register
Using the TUI
- wh open TUI
- layout <name> change layout, where <name> can take a superset of:
  - src source, command
  - split source, assembly, command
  - asm assembly, command
  - regs add registers to current layout
- focus <name> set main focus to <name> sub-window, similar to layout above
- wh <name> <count> change window height of <name> by <count>, where count can be positive or negative
- tui disable or <C-x> a disable the TUI

Looking around (tips)

Examine stack
- info frame print human-readable information about the active stack
- p ($ebp-$esp)/4 + 2 print the “length” of the stack necessary to see the stack down to the saved ebp and ret addr
- x/16xw $esp print the full stack, where 16 is the output from the above command
- x/2xw $ebp print saved ebp and saved ret addr
Examine text segment
- info line info about the current line of the active frame
- disas print dissembled x86 instructions starting at eip
- disas /r $eip,+24 print 24 instructions, including each instructions specific bytes
- disas /s $eip,+24 print 24 instructions, including relevant source code
Printing information
- display/<format> <expr> to display <expr> after every command to gdb

Breakpointing

break <linenum> break just before <linenum> executes in the current file
break <filename>:<linenum> break just before <linenum> executes in <filename>
until <linenum> step until source’s <linenum> is reached, or would exit current stack
watch <expr> to break when the evaluated value of <expr> changes
- set can-use-hw-watchpoints 0 may have to enable software-only watchpoints (to function properly on a VMWare VM?)
delete display clear all auto-displays

Editing memory

Can use either print or set to set values/edit memory during debugging
set reqpath = "foo" where reqpath is a char*, sets the reqpath string to "foo"
set {int}0xbffff59c = 0x10 set the memory at address 0bxffff59c to the value 0x10

Miscellanea

If some symbols seem to be missing, check the compiler’s optimization level
set $esp += 4 increment $esp register by 4
define <command> define a user alias to <command>
target record-full record all actions so can reverse debug
Debugging threads
Debugging forks: set follow-fork-mode child
Specify a location: br * <addr> break at addr

Security

Overview

Overview of cryptography
Awesome cryptography
Textbooks
- Introduction to Cryptography
- Handbook of Applied Cryptography
Cryptographic key
- Key types
TinySec, an example use-case with great explanations of some cryptographic topics

Information security

Security tools

Practical online security
- Password strength checker (entropy) (time to crack)
- Strong password generator (Diceware)
- 2FA: set up two-factor authentication for important websites
- Firefox Monitor: see if you’ve been part of an online data breach
Additional tools
- Browser fingerprinting (AmIUnique)
- Tor (wikipedia)
  - How HTTPS and Tor work together
- Keybase
- Qubes OS
- Keychain Access (MacOS)
  - Identity preferences map mTLS client certificates to domains

ASN.1

Abstract Syntax Notation One is an interface description language (objects-only) allowing data structure definitions in support of platform-agnostic serialization, used extensively in cryptographic applications
- ASN.1 specifies abstract objects
- Encodings translate the abstract objects into a stream of bytes; DER, a subset of BER, gives a unique encoding value to each ASN.1 object, a useful quality for cryptographic purposes (all encodings)
- Encoding of particular object generally follows the type-length-value (TLV) scheme
- Primitive types list
Complete information 🌟
Resources (about) (intro) (specification intro) (cheat sheet)
Playground
- Denote octet string values as 'deadbeef'H
Object ID (OID) repository
pyasn1: an ASN.1 parser written in Python (github) (tutorial)

OpenSSL

Home (wikipedia)
Common commands (more 1) (more 2)
OpenSSL Cookbook
Common file types
- .pem Privacy Enhanced Mail, a Base64-encoded bucket, where file contents are indicated in the header/footer lines
- .key any type of key, usually a private key
- .csr certificate signing request
- .pfx PKCS #12, a generic archive format; often: encrypted and signed bucket containing a private key and associated X.509 certificate/chain
- Common certificate filename extensions: .pem, .cer, .crt, .der, .pfx
Common tasks
- Set up a CA (more)
- Implement mutual TLS authentication (mTLS)
Snippets
- openssl pkcs12 -in admin_operator.pfx -nocerts -nodes -out out.key extract private key from PKCS #12 (or -nokeys to extract just certificates) (man) (dealing with bag attributes)
- openssl rsa -text -noout -in out.key examine RSA key (man) (more)
- openssl x509 -text -noout -in certifier.pem examine X.509 certificate (man)
- openssl req -noout -text -in controller.csr examine PKCS #10 certificate signing request (man)
- openssl verify -CAfile ca.pem -untrusted intermediate.cert.pem cert.pem verify certificate chain
- openssl asn1parse -i -in out.crt examine ASN.1-parsed fields
- openssl s_client -CAfile /var/opt/magma/certs/rootCA.pem -connect bootstrapper-controller.magma.test:9444 debug SSL handshake (man)

Let’s Encrypt

Overview
Getting started
Certbot automatically generate TLS certificates

Security primitives

Message authentication code (MAC): symmetric analog to the (asymmetric) digital signature; a keyed hash that uses shared symmetric key to tag/validate the conveyed message to provide message integrity and authentication
Block cipher modes (about) (specs) (discussion)
- Confidentiality modes: ECB, CBC, CFB, OFB
- Authentication mode: CMAC
- Authenticated encryption mode: CCM
- High-throughput authenticated encryption mode: GCM
- Methods for key wrapping: KW, KWP, TDEA
- And a few others
Hash chain: successive application of one-way hash function to a piece of data
- In password storage/authentication: server computes and stores h^1000(password), and user successively authenticates by sending h^999(password), h^998(password), …

Keying

Key exchange: sharing secrets between two or more parties
- Group key: key shared between group of users, typically for individual decrypting of a broadcast message
- Pre-shared key (PSK): secret shared between parties before first use
  - TLS can use pre-shared keys to establish a TLS connection
  - PSKs are associated with an identity (a UTF-8-encoded string), so e.g. a client would store its own (ID, PSK) pair while a server would store multiple (ID, PSK) pairs
Key derivation function (KDF): derive one or more secret keys from some input value, e.g. to generate session keys from a master key or a symmetric key from a password

Public-key cryptography

PKI overview
Discrete logarithm problem: deciding S given (g, p, g^S mod p) (generator, prime, key fragment) being a hard problem
Diffie-Hellman key exchange (DH)
- Diffie-Hellman considered a “public-key technology” since it has public and private components; the algorithm itself doesn’t encrypt anything
- Using ephemeral keys with Diffie-Hellman allows for perfect forward secrecy because the “private” DH keys are stored only for the duration of the session, then discarded (in fixed DH, the public parameter is embedded in the node’s certificate)
Perfect forward secrecy
- Consider all past messages recorded, available, and algorithms fully specified: could the sender/receiver collaborate to recover the plaintext of these past messages? If yes, then the strategy does not allow for PFS
- Assume that anything sent using long-term public keys (RSA public key, DH public parameter, etc.) is cleartext (description)
- Focus is the ephemerality of the DH keypairs (which are used to generate the session key), rather than ephemerality the session keys (which are always ephemeral)
- Example of non-forward secret strategy: generate a session key, encrypt it with receiver’s public key, and send to receiver (key transport)
- DH vs. RSA for PFS
- Key-agreement protocols where both parties influence the final derived key are the only way to implement PFS
Key encapsulation: method for choosing an asymmetrically-transported symmetric key which doesn’t require the use of a padding scheme
- Optimal Asymmetric Encryption Padding (OAEP), a padding scheme often used with RSA encryption
Public key fingerprint: encode key into bytes, hash it, then pretty print it as e.g. a hexadecimal string; produces a shorter, human-readable version of the key (and optional additional data like name of key holder)
Public key certificate: node’s public key, identifying information, and digital signature of a relevant certificate authority (file formats)
Digital signature: first hash the message (e.g. a certificate’s identifying information) into a digest, pad, then sign with the private key (e.g. by raising to the private key, mod an appropriate N)
- A digest is the output of a hash function, descriptively relevant when used to verify a fingerprint without recovering e.g. the fingerprint’s source key
- Key signature is one party signing another party’s key fingerprint
- Signatures can also be blind, using a blinding function first before signing
Public key infrastructure binds public keys to identities using certificates issued by a certificate authority
- CA (certificate authority): store, issue, sign certificates
- RA (registration authority): verify identities of entities requesting certificates
- VA (validation authority): verify that a certificate is valid, e.g. by distributing certificate revocation lists (CRLs) or implementing the Online Certificate Status Protocol (OCSP)
Hybrid cryptosystem combines functionality of public-key with efficiency of symmetric-key
- Key encapsulation scheme uses public-key
- Data encapsulation uses symmetric-key
- Majority of cryptosystems use this method–e.g. TLS, OpenPGP, etc.
Pretty Good Privacy (PGP)
- Hybrid cryptosystem where symmetric “session” key is the only data encrypted with receiver’s public key
- Follows OpenPGP open specification
Post-quantum cryptography
- Supersingular isogeny key exchange: post-quantum analog to Diffie-Hellman

Physical security

Frequency hopping implementation (didactic)
Crypto processor
- Acceleration of encryption
- Physical tamper protection
- Encryption over bus/in RAM
- Encrypted I/O
Crypto-shredding: “delete” encrypted data by zeroizing the encryption key(s)
Cold boot attack: RAM holds its values in memory for seconds to minutes after power has been removed, so data/keys can be read from these memory locations after a cold (full/hard) reboot (video)

Miscellanea

Code vs. cipher: code encrypts by word/phrase, cipher encrypts by letter
Secret sharing: give each of n people each a different point on a (k-1)-degree polynomial; any k of these points can reconstruct the original polynomial, where the secret is the polynomial’s constant term
- More general version is threshold cryptography, where encryption/decryption/other operations require some threshold number of parties to complete
Cryptographic Message Syntax (CMS) (spec)

Networks

Intro

Topology

Network topologies
Ad hoc network: wireless network not relying on pre-existing infrastructure (routers in wired networks, access points in wireless networks)
- Mobile ad hoc network (MANET): nodes are freely moving, so each node’s view of the network topology is dynamic

Routing

Routing overview
Multi-hop routing allows network coverage to extend beyond the range of a single node by using neighboring nodes as relays; requires some form of routing protocol
Unicast: one-to-one, with a single node logically communicating end-to-end with a single other node across the network, e.g. on the Internet
Broadcast domain: logical division of a network in which all nodes are reachable by broadcast, e.g. a local network
Multicast: addressed to a group of destination nodes–can be at application layer or network-assisted
Anycast: one-to-any, utilized in applications where the contacted node can be any from a relevant group, e.g. CDNs or the DNS

Proxying

Forward vs. reverse proxy

Internet

TCP/IP pocket reference
Overview of TCP/IP protocols and the Internet
TLS: Transport Layer Security
- TLS abstraction layer: it doesn’t fit well into either OSI or TCP/IP model, but effectively sits at the bottom of the application layer
- Every byte of a TLS session explained (TLS 1.2) (TLS 1.3)
High Performance Browser Networking
Private IPv4 addresses
- 10.0.0.0 - 10.255.255.255.255
- 172.16.0.0 - 172.31.255.255
- 192.168.0.0 - 192.168.255.255
How multiple clients connect simultaneously to one port: a TCP connection is identified by a 4-tuple of (source address, source port, destination address, destination port)
Local/home network setup
- Access home network anywhere with dynamic DNS
- Forward ports on home router
Submarine communications cables (intro) (history) (overview) (security)
GRE tunnels (wikipedia)
Deep packet inspection includes inspection of headers and data inside the IP header by devices other than the intended endpoints
- Firewalls in general can perform deep packet inspection, identifying network flows by inspecting application data
- Stateful firewalls perform shallow inspection by examining only the transport-layer headers
- nDPI, an open source library for performing deep packet inspection (list from Wikipedia)
DNS (Domain Name System), a full query takes ~100ms total
NAT (network address translation) translates IP headers between public-network IP tuples and private-network IP tuples
- NAT traversal
  - STUN connects two peers in most cases (control plane relay)
  - TURN uses public relay servers to forward data between endpoints, as a fallback when other techniques fail (data plane relay)
  - ICE recommendations for generalized solution to NAT traversal
- NAT features
  - Mapping
    - Endpoint-independent: easy, a private source IP will have the same public source IP independent of destination IP
    - Endpoint-dependent: hard, a private source IP can have different public source IPs depending on destination IP
  - Firewall
    - No: incoming packets never filtered
    - Yes: incoming packets always filtered
    - Stateful: incoming packets conditionally filtered, only allowed if paired to a recently outgoing packet to same destination IP:port
  - Other
    - Double NAT: multiple layers of address translation
IPv4 census map (2013), using a space-filling curve to translate into 2D space

Local networks

127.0.0.1 vs. 0.0.0.0: local-only vs. “every available network interface”
Virtual LAN: use tagging to partition a single physical LAN into multiple broadcast domains
Data link layer vs. medium access control layer: medium access control is a sublayer within the data link layer, providing flow control and multiplexing for the transmission medium
802.11 vs. Wi-Fi
- 802.11 is the standard specified by the IEEE
- Wi-Fi is is an 802.11-based certification granted from the Wi-Fi Alliance

OSI model

Non-TCP/IP layers
- Session and presentation layers encompassed by TCP/IP’s application level
- Session layer (5): authentication, authorization, etc., similar to HTTP’s cookies
- Presentation layer (6): normalization of data, such as serialization and encryption
Application level can handle UDP-like packet loss, with e.g. RTSP
Model overview
OSI protocol data units
OSI vs. TCP/IP model
Frame encapsulation in TCP/IP model
As it applies to the Unix domain

Tools

Miscellanea
- List of traffic monitoring tools
Curl
- Man: transfer a URL
- Tutorial
- HTTPie, a modern alternative
- Snippets
  - curl -v for verbose curl output, including headers
ZeroMQ
- Guide
- zerorpc: RPC framework built with ZeroMQ and MessagePack
Tcpdump
- Man: dump traffic on a network
- Tutorial
- Snippets
  - tcpdump -D list available network interfaces
  - tcpdump -i lo udp port 49183 -vv -X sniff UDP packets on local/loopback 49183 port
  - tcpdump -i any port 8080 sniff traffic to port 8080 on any interface
  - tcpdump -i lo port 2461 -w dump.pcap sniff packets sent to port 2461 from and to the local device; and write to dump.pcap
  - mergecap *.pcap -w merged.pcap merge pcaps
Netcat
- Man: arbitrary TCP and UDP connections and listens (an alternative to telnet with better scripting and error messages)
- Tutorial (DigitalOcean)
- Getting working on Mac by reconciling shipped vs. Homebrew netcat
- Find IP address on a Mac
- Port scanning, and other netcat use-cases
- Reverse shell
- Snippets
  - nc -l 8080 listen on port 8080
  - nc -l 8080 < out.html serve out.html over port 8080
  - nc localhost 8080 connect to 127.0.0.1:8080 over TCP
  - nc -u localhost 8080 connect to 127.0.0.1:8080 over UDP
  - nc localhost 8080 > out.html receive to out.html from 127.0.0.1:8080
Socat
- Man: construct a bytestream, with many options for source and sink types
- Intro
- Examples
  - Local UDP broadcast server
    - ip addr find broadcast address for desired network interface (can also use ifconfig)
    - socat -u udp-recv:8080 - start receiver
    - socat - udp-sendto:127.255.255.255:8080,broadcast start sender

Simulation

Network simulators and emulators can be used to simulate network topologies and events
- ns-3
- Mininet (example) (walkthrough) (intro) 🌟
- Mininet-WiFi (example)
- GNS3
More generically, event-driven simulation frameworks support broader simulation needs
- SimPy (overview)

ONC RPC

Overview

Aka Sun RPC, developed in the 1980’s by Sun Microsystems (which was later aquired by Oracle)

Included in glibc under the header <rpc/rpc.h>

Source code and documentation in glibc/sunrpc with:

git clone git://sourceware.org/git/glibc.git
git checkout --track -b local_glibc-2.27 origin/release/2.27/master

Uses XDR for serialization
- Specification (Oracle)
- Reference (IETF RFC)
- Python provides xdrlib for high-level packing/unpacking (example)

References

Commands

rpcbind must be running on the server’s machine. rpcbind records and resolves mappings from RPC program numbers to universal addresses.
- Client first connects to server machine at port 111, and must request the universal address for that program number on that server. It then makes a new call to the specific universal address.
- The mapping is from {RPC program number, version of that RPC program, requested transport protocol} -> {TCP/UDP port}
- Universal addresses in IPv4 refer to the {IP address, port number} pair
- Earlier versions of rpcbind were called portmap
rpcinfo report RPC info such as mapped programs and transport types
- rpcinfo -p relevant ports
- rpcinfo -s concise report
rpcgen -C -N -a add.x generate stubs from add.x specification
- Tutorial
- -C ANSI C stubs
- -N C-style declarations (allows e.g. multiple arguments passed per stub)
- -a create example usage files
nmap -p - localhost scan listening ports (e.g. to ensure rpcbind is running)
netstat -tulnp list listening sockets
nc -u localhost 49183 send UDP packets to localhost port 49183
tcpdump -i lo udp port 49183 -vv -X sniff UDP packets on local/loopback 49183 port (use rpcinfo -p to determine port)
- How to read UDP packets
- tcpdump intro
- -w dump.pcap to dump to file readable by Wireshark, which can understand the RPC protocol (may have to set a Wireshark option or two)

Miscellanea

Converting to e.g. opaque data type:
- Use XDR utility functions such as xdr_bytes()
Constantly bad packet checksums:
- Probably caused by hardware offloading
- Can disable hardware offloading if desired
Headers
- IPv4 minimum 20 bytes
- UDP 8 bytes
Synchronous vs. asynchronous RPC’s

Athena workstations at MIT

General

How to log in remotely
Full list of applications
The Kerberos protocol (intro) (eli5) (wikipedia) (full tutorial)

How the filesystem works

All about AFS
add and attach commands
- attach attaches the specified locker
- add does the same but also edits $PATH and $MANPATH
An Athena locker is a directory on the Athena filesystem

Using the VPN

Useful to e.g. access academic papers released only to people on MIT’s network. - Why MIT’s VPN is great - Install the VPN software: Cisco’s AnyConnect VPN - Log in to the VPN with Duo: VPN address is vpn.mit.edu/duo

We want to be able to log in easily–that is, without providing a password. We’ll use the Kerberos protocol. - Install Kerberos Extras to allow creation of a Kerberos ticket - Acquire a Kerberos ticket using the Ticket Viewer app. Alternatively, use the kinit family of programs: - kinit <athena-username> get Kerberos ticket - klist list tickets - kdestroy destroy tickets - Add the following block to your ~/.ssh/config file. This will allow login with ssh athena, and will automatically use local Kerberos tickets.

```sshconfig
Host athena
    HostName athena.dialup.mit.edu
    User <athena-username>
    GSSAPIAuthentication yes
    GSSAPIDelegateCredentials yes
    StrictHostKeyChecking no  # optional, fixes an issue I kept running into
```

Miscellanea
- Kerberos config file is generally stored at /etc/krb5.conf, though on Macs it may be at /Library/Preferences/edu.mit.Kerberos
- Kerberos on Mac OS X
- Full list of SSH config options
- GSSAPI-based authentication

Better remote editing

We want to be able to edit files in our Athena workspace using the tools on our local machine. - Common file-sync/remote editing tools: SSH, SCP, rsync - File transfer overview and tools - SSHFS, generic file-transfer solution for editing files on an Athena directory from your local machine - OpenAFS is an AFS-specific alternative to SSHFS, and one officially supported by MIT - Intro - How to install OpenAFS on Mac - Install AuriStor’s implementation of AFS, as is recommended by IS&T (MIT’s cell is athena.mit.edu) - Optionally allow usage from menu bar - Log into AFS: kinit <username> && aklog - Commands that touch many files are slow over mounted remote filesystems, so if your editing environment makes calls like git status this will slow down the editing environment and make it appear sluggish. Instead, we could use rsync to copy files over automatically. Sample rsync script: - 824-sync pull copy remote files to local machine one time - 824-sync push copy local files to remote machine on changes - Make sure you performed the Better remote login steps above–this won’t work without a Kerberos ticket, etc. - May need to install fswatch, e.g. brew install fswatch

```sh
# Store this in a script such as `~/.scripts/824-sync`

# Usage: 824-sync <push/pull> [-f]

# This script syncs the `src` directory of my 6.824 work from the local machine
# to the Athena machine.

# This won't delete files on the Athena machine.
# It also won't overwrite "newer" on the receiver unless you pass `-f`.

shared_dir="src"  # shared dir across local, remote
src_dir="/Users/Hunter/Desktop/mounts/${shared_dir}"
dest_dir="/mit/hcg/6.824/${shared_dir}"
exclude="*.swp"

# Return early if no $1
[ -z "$1" ] && echo "Usage: <push/pull> [-f]" && exit 1

# Don't protect newer files on receiver when -f passed
[[ "$2" = '-f' ]] && u='' || u='u'

rsync_push="rsync -az${u}P --exclude=${exclude} ${src_dir}/ athena:${dest_dir}"
rsync_pull="rsync -az${u}P --exclude=${exclude} athena:${dest_dir}/ ${src_dir}"

# Just pull and exit
if [ "$1" = 'pull' ] ; then
    echo 'pulling...'
    $rsync_pull
    exit
fi

# Continuously push
echo 'pushing...'
$rsync_push
echo 'watching...'
fswatch -or "$src_dir" | while read ; do $rsync_push ; done
```

Projects

Twilio-Dropbox integration

About
- Create Dropbox text file with contents of SMS messages
- Gist of the Twilio function
References
- Twilio functions (dashboard) (debugger) (intro) (docs)
- Twilio dashboard (project info) (configure numbers)
- Dropbox JavaScript SDK (docs) (npm)
Description
- Whitelisted phone numbers send SMS messages to a Twilio-managed phone number
- SMS message contents are placed into a Dropbox file named todo_yyyy-mm-dd_hh-mm-ss_epoch.txt
- Returned SMS message is either a link to the created file or a summary of the Dropbox error message

EOD toy robot

About
- Toy version of an explosive ordnance disposal (bomb disposal) robot and control unit, inspired by TALON
- Want to control the robot and receive video data over a secure channel
Components
- Chassis: Tracked Robot Smart Car Platform
  - Dual DC 9V motors, PWM control, Hall effect feedback
  - Self-assembly: 1-2 hours (guide)
  - Motor wiring
- Batteries: Panasonic Rechargeable Li-ion 18650
  - Charger
- Switch: Gardner Bender Electrical Toggle
- Motor controller: DROK 16A DC Brush Motor Controller
  - Could have used a smaller controller
- Voltage regulators: DROK 5v Regulator
  - Variable, but used 8V to 5V
  - Reference sheet
- Board: Raspberry Pi
  - Raspberry Pi 3 Model B
- Board peripherals: (USB to TTL serial cable) (microSD reader) (microUSB power cable)
- Cameras: (picamera) (thermal)
- Touch screen: Landzo 7 inch capacitive touch screen
  - Howto: add few lines to config.txt; requires driver for touchscreen capabilities
- USB controller: ZD-V+ USB Wired Gaming Controller
  - Used Python’s evdev module to parse controller input events

Screen

Cheat sheet
Manual
Commands
- screen /dev/ttyUSB0 115200 -S session-name start new session-name at 115200 baud on ttyUSB0
- screen -ls list sessions
- screen -x attach to some session
- <C-a>d detach from session
- screen -r session-name reattach to session-name
- screen -X -S session-name quit kill detached session-name
- <C-a>\\ kill attached session

Tmux

Cheat sheet
Terminology
- Tmux window has multiple panes
- Vim tab is split into splits
Commands
- Command line
  - tmux ls list sessions
  - tmux a attach to some session
  - tmux a -t session attach to session
  - tmux kill-session -t session kill session
- Command mode
  - :set -g mouse on enable mouse, enable scrollback
  - :set-window-option mode-keys vi, then prefix-[ enable vim-like page scrolling
- Pane manipulation
  - prefix-% new pane, vertical split
  - prefix-" new pane, horizontal split
  - prefix-o next pane
  - prefix-<space> cycle pane layouts
  - prefix-q show pane numbers
  - prefix-} swap panes
  - prefix-[ enable page scrolling
  - prefix-x kill pane
- Window/session manipulation
  - prefix-c create new window
  - prefix-d detach

Vim

Intro

Miscellanea

Regex in Vim (and :h usr_27.txt)
- s/pattern/string/cgiI
  - I don’t ignore case for the pattern
- foo \zsbar\ze baz partial match (lookbehind/lookahead)
GDB debugging in Vim
Scripting in Vim
- Vimscript overview
- Python scripting (man)

UI

hi SignColumn ctermbg=none set background color to transparent for the sign column
Syntax highlighting
- :syntax list syntax items for current buffer
- :syntax match mkdBlockquote "\[.\]" add new syntax item

Motion

Sentences
- d) delete to end of the sentence
- das delete a sentence
- cis change inner sentence
]m, [m go to start of a method (e.g. a def in Python)
gv reselect previous visual selection
W, E, B operate on WORDs, i.e. only whitespace breaks words
:lne go to next item in quickfix list
mM global, persistent mark M
% can switch between logically grouped C preprocessor macros
- Requires using matchit.vim extension
- Can include with runtime macros/matchit.vim in vimrc
c-y, c-e scroll screen up, down one line

View

:cq quit with non-zero exit code
:%s/\[x\]//gn print number of matches of [x]
store\|network1 search store or network1
:v/INACTIVE\|ACTIVE/p show all lines not matching
c-w x swap splits
:f current filename
:tabm # move current tab to # location, or relative (-1, +1, etc)
tabdo, bufdo, windowdo : do some command per unit
- tabdo set nolist disable showing listchars in all tabs
:echo expand('%:p') print full path of current file
:pwd, :cd .. change working directory
3gt jump to tab 3
Redirect ex command output, to register (@a) or file (>out.log) vimscript :redir @a :g/./p :redir END ```
Buffers
- [b and ]b to quickly switch buffers (requires unimpaired.vim)
- :Buffers to fzf-jump to buffers (requires fzf.vim)
- :e edit new file
- :new create new unnamed buffer (also :enew, :tabnew)
- :bd delete current buffer
- Work around Cannot write, 'buftype' option is set error via set bt=
Folds
- za toggle current fold (zc, zo to close, open)
- zA toggle all folds related to cursor position (zC, zO to close, open)
- zR open all folds
- zf create fold (in manual mode)

Edit

:'<,'>!tac reverse order of selected lines (may have to use gtac)
:r insert contents of (named) file below cursor
sort u on selection sorts the selection while removing duplicates
sort /./ sort, but skip first character on each line
:%!sort -k3nr sort buffer by 3rd column, as numbers, reversed
vim rsync://athena/6.824/src/ see directory in athena, then can directly edit a file
<c-r>a paste register contents from register a
qaq:g/Wrap/y A yank all lines containing Wrap to register a
:%S/old/new/g search/replace with case preservation (requires vim-abolish)
:%s/^\W*\zs\w/\U&/g capitalize first letter of every line
:%s/$\(^\(\s*-\s$\?\)\|$\w[.?!] $\)\zs\w/\U&/g janky attempt to capitalize the first word of every sentence
:%s/Error$f$\@=/Fatal/g change all Errorf to Fatalf via positive lookahead regex
<c-v><tab> add literal tab character from insert mode
g/SKIP because it is allocated with 0 resources/d delete all lines matching the regex
v/FREEZE CANDIDATE because it passed initial checks/d delete all lines not matching the regex
Spell check
- :set spell enable
- :set nospell disable
- ]s next error, relative to cursor
- [s previous error, relative to cursor
- z= find suggestions for word under cursor
- zg add word under cursor to “good words”
- zug remove word under cursor from “good words”

Troubleshoot

:map debug key mappings
Trace slowdown source
- vim -u NONE start Vim without a .vimrc; if this fixes the slowdown, the issue is in the .vimrc

Profile

Profile normal usage to find bloated functions (see end of the file for a sorted list of time-consumers)

:profile start profile.log
:profile func *
:profile file *
" At this point do slow actions
:profile pause
:noautocmd qall!

Clean up

Check discrepancies between expected packages and actual imported scripts:

" Check installed packages (assuming using Vundle)
:PluginList
" Check imported scripts
:scriptnames

Plugins

vim-unimpaired
- ]b next buffer
- ]q next error
- ]c next error
- ]e exchange line below
- [e exchange line above
- ]s next spelling error
fzf
- :BLines lines in current buffer
- :Commands colon commands
- :GFiles fzf based on .gitignore
- :History/ search history
- :History: command history
- :Buffers buffers
- :Lines lines in loaded buffers
- :Maps see all mappings
- :Marks saved marks
- :Tags tags in the project
Swap
- g>, g< swap arguments right or left
- gs interactive mode for argument swap
  - s, S sort, unsort
  - r reverse
  - u, <c-r> undo, redo
  - g, G group, ungroup
  - 3 go to 3rd argument
textobj-latex
- ae, ie environment \begin{...} to \end{...}
- aQ, iQ double-quoted text
- a$, i$ inline math surrounded by dollar signs
UltiSnips
- :UltiSnipsEdit edit snippets for current filetype
- :h UltiSnips-interpolation help with interpolation options
- Snippet definition formats
  - snipMate format: snippets/*
  - UltiSnips format: Ultisnips/* 🌟
- Selected UltiSnips options
  - b beginning of line: expand only if the trigger is the first non-whitespace word on the line
  - i in-word expansion: snippet can be triggered in the middle of a word
  - r regular expression: trigger is expected to be a Python regular expression
  - t keep tabs: do not expand tabs
  - A automatic: snippet will be triggered automatically
- Example UltiSnips snippet definition
```
# ~/.vim/UltiSnips/tex.snippets

# Automatically expand when `\beg` is typed
snippet \beg "begin{} / end{}" bA
\begin{$1}
$0
\end{$1}
endsnippet
```
- Community-maintained snippet definitions

Visual Studio Code

Keyboard shortcuts
- when clause contexts

Visualize C call graphs on a Mac

Two-liner

gcc-7 -fdump-rtl-expand <*.c>
perl <path/to/egypt> <rtl-file> | dot -Tpng -o out.png && open out.png
Also consider Sourcetrail, the interactive source code visualizer

Summary

Compile with gcc-7 flag -fdump-rtl-expand
Convert the outputted RTL file to a graph specification with egypt
View the graph as a PNG with dot

Alternatives

See this Stack Overflow answer for a list of alternatives to this approach.

Compiling with RTL representation

brew install gcc
gcc-7 -fdump-rtl-expand <*.c>

As a precursor, egypt wants to use output from GCC, while, apparently, Macs install clang and call it gcc. To fix this, brew install gcc and use gcc-7 instead of gcc.

RTL representation, the Register Transfer Language, is a low-level intermediate representation used during the last part of compiler work. Egypt uses this representation to create a graph-based call graph. The -fdump-rtl-expand flag dumps the RTL representation for Egypt to use.

Installing graphviz and egypt

brew install graphviz
# Download egypt from http://www.gson.org/egypt/
cd <download-location>
perl Makefile.PL
make && make install

Generate visual

perl <path/to/egypt> <generated-rtl-file> | dot -Tpng -o out.png
open out.png

Dot is a directed graph visualizer. It takes the graph-specification output from egypt and here outputs a PNG.

👨‍💻 Professional notebooks 👨‍🚀

Finances

Salary information

Equity compensation

AWS

General

VPC

EC2

IAM

Kubernetes

General

API server

Scheduling

Node runtime

Scale

DNS resolution

EKS

Docker

Containers

Reference

Snippets

Tcpdump

Cloud

Interviews

Overview

Interview types

Tools

Remote-friendly

Prep

Rubrics

Coding

Design

Behavioral

System design

Specific problems

Partitioning and replication

Latency

Software engineering

Overview

Decision making

Development

User experience

Collaboration

Patterns

Culture

Concurrency

Typing

CI/CD

Package management

Computation structures

Tools

Libraries

SSH

IntelliJ

Serialization

Intro

Some formats

Canonical forms

Literature

Catholicism

Games

Traditional sports

Factorio

Scrabble

Housing

General

Linear algebra

Resources

Topics

Julia

Reference

Snippets

Jupyter notebooks

Strength training

General

Form

Fixes

Macro tracking

Kinesiology

Shell